Sharpening GIS at Your Will

www.gisinternals.com is down

2011-11-26T01:06:00.000+01:00

The GDAL and Mapserver daily build package provider site www.gisinternals.com/sdk is currently not available due to server relocation. I hope it will be online again by the end of this weekend.

Windows SDK support site has been relocated

2011-04-19T13:45:00.000+02:00

Looks like my account on dyndns has been lost :-( Since I'm getting tired to update the account in every month, I've already registered a domain to point to this location which is:

http://www.gisinternals.com/sdk/

If you have any links or bookmarks to point to the support site then update them to refer to this new location. The old URL is considered to be deprecated from now.

GDAL installers for multiple Python versions

2011-02-03T22:58:00.000+01:00

After a bit of tweaking in SWIG and Python distutils the installers for GDAL are now available targeting Python 2.6/2.7 and 3.1 (x86 and x64) at http://vbkto.dyndns.org/sdk/

Hmmm... I must mention that quietly, there haven't been made an effort to fix the problem descibed in this post with the recent Python versions. Probably I'm the only one who's trying to compile anything with MSVC2010? I doubt it. Looking forward to the next version...

Windows binaries for GDAL 1.8.0 and MapServer 5.6.6

2011-01-24T23:47:00.000+01:00

As you may already know both GDAL and MapServer had a new stable release in the recent days and I've updated http://vbkto.dyndns.org/sdk/ to provide the Windows binaries according to these versions.
If you prefer using the latest stable releases (GDAL 1.8.0 and MapServer 5.6.6) then choose the 'release' packages, but you may also download the most recent daily builds of the stable branches (always containing all the subsequent changes since the last release) by choosing the 'stable' packages.
The development versions of GDAL and MapServer are also available for testing by using the 'development' packages.
I've also added Windows installers for the GDAL components providing to install the GDAL command line tools and the python bindings easily. Installers for MapServer with IIS registration and PHP support will also be available shortly.

Daily built binary packages for MapServer with OpenSSL support

2010-12-22T21:18:00.000+01:00

SSL support has been added to the daily built binaries which provides the option to access a secure WMS/WFS server on Windows. Here are the key steps to configure this option in your Mapserver/Mapscript installation:

1. Configure the mapfile with one or more WMS/WFS client layer

This will require to configure a layer with 'CONNECTIONTYPE WMS' or 'CONNECTIONTYPE WFS' and set up your connection information according to the following fragment:

LAYER
    CONNECTION "https://www.secureservice.com/service?"
    CONNECTIONTYPE WMS
    METADATA
      "wms_srs"    "EPSG:4326"
      "wms_name"    "NAME"
      "wms_format"    "image/png"
      "wms_server_version"    "1.1.1"
    "wms_auth_username" "username"
    "wms_auth_password" "password"
END
    TYPE RASTER
    ...
END

2. Make sure to install all the required files in your deployment

For the SSL support you require to install libeay32.dll and ssleay32.dll along with libcurl.dll in the run-time configuration. Without having OpenSSL added to the builds you got the following error:

msHTTPExecuteRequests(): HTTP request error. HTTP: request failed with curl error code 1 (Protocol https not supported or disabled in libcurl)

3. Set up the cerificate bundle along with your installation.

In short it will require to set up the CURL_CA_BUNDLE environment variable to point to the location of the .crt file. You can find this file (curl-ca-bundle.crt) in the /bin/curl folder of the binary packages mentioned above. The file contains the most recent version of http://curl.haxx.se/ca/cacert.pem (at the time of the nightly build).

Note: CURL_CA_BUNDLE can be set up system wide (in My Computer->Properties), however you may prefer to apply this environment setting only for the process hosting the mapserver libraries so as not to cause side effects for other applications using libcurl and openSSL. The main issue here is that most of the libararies use getenv to retrieve the environment setting, which operates only on the data structures accessible to the run-time library (msvcrt) and not on the environment "segment" created for the process by the operating system. In this regard the libraries work only on a snapshot of the variables that have been set during the process startup. In this regard you may choose to set up the environment before the process has been started (by using a startup script) or use _putenv to set up the environment variable at run time. I will be trying to discuss this topic in more detail in a subsequent post.

You may check whether your WMS/WFS server is working with the standalone installation of curl.exe in the binary packages and open an SDK command prompt by using SDKShell.bat supported with the packages. In the command prompt you may use a valid url to your service, like:

curl "https://username:password@www.secureservice.com/service?..."

At this point if you get valid data response you may omit the following step.

4. Extract the certificate entry from the server

In the previous step if you get the following error, your remote server probably use a self-signed SSL certificate and the server certificate is not included in your CA bundle file.

curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

To get the remote server certificate you may use openssl.exe in the command prompt (also included in the daily buillt packages) according to the folowing example (you may probably use the default SSL port:443 in most cases):

openssl s_client -connect www.secureservice.com:443

Copy all from "-----BEGIN CERTIFICATE-----" tag to "-----END CERTIFICATE-----" tag. Paste it at the end of the curl-ca-bundle.crt file, and repeat #3 to make sure the certificate data is now working.

OGR MS SQL 2008 Spatial driver

2010-09-12T16:30:00.000+02:00

I've now added the initial revision of the OGR MS SQL Spatial driver to the GDAL repository. For further information please refer to the driver information page.
Working binaries (which always contain all the recent fixes) can be downloaded from the daily build system available at: http://vbkto.dyndns.org/sdk/

Since I didn't have the chance to test with all kind of the fancy spatial data types any feedback on the potential issues are graciously accepted.

Upon the experiences gathered when writing this driver, I'm also planning to add support for accessing the native SqlGeometry/SqlGeography serialization format in the MapServer sql2008 driver which appears to provide a significant increment in the performance of the data retrieval.

Python distutils and manifest files (Visual Studio 2010)

2010-05-02T22:19:00.000+02:00

Visual Studio 2005 introduced a new deployment model for Windows client applications based on isolated applications and side-by-side assemblies. The redistributable libraries (such as MFC, ATL, CRT) have been rebuilt as shared side-by-side assemblies and installed in the native assembly cache, WinSxS folder in Windows. This has provided a common location for particular versions of the dll-s and the applications had to declare their dependencies by using a manifest file. The manifest file is an XML file which contain references to the dependent dlls located in WinSxS, these files had to be deployed along with every dll or executable which depend on the redistributables. Without having the manifest (either by a separate file or embedded in the executable as a resource) the application generates an error.
Though the purpose of this change was to simplify deployment, the result was probably the opposite. As a result Microsoft changed deployment requirements in Visual C++ 2010. As of Visual Studio 2010 Microsoft has backed out the manifest generation and no manifests are generated by default with the VC2010 builds. There's no requirement to embed manifests in the executables anymore and CRT-s need not be installed in WinSxS. All you need to do is copy the VC++ dependent DLLs to the application folder and run.

This change may however cause issues for those build systems (like python distutils) which don't expect that the manifest may eventually be missing and stop the compilation in this condition. I don't see any change in this regard in the python SVN trunk either. Fortunately to prevent from this issue we could easily alter the corresponding file (/Lib/distutils/msvc9compiler.py) to something like:

   # embed the manifest
   # XXX - this is somewhat fragile - if mt.exe fails, distutils
   # will still consider the DLL up-to-date, but it will not have a
   # manifest. Maybe we should link to a temp file? OTOH, that
   # implies a build environment error that shouldn't go undetected.
   if os.path.isfile(temp_manifest):
        mfid = 1 if target_desc == CCompiler.EXECUTABLE else 2
        out_arg = '-outputresource:%s;%s' % (output_filename, mfid)
        try:
             self.spawn(['mt.exe', '-nologo', '-manifest',
                  temp_manifest, out_arg])
        except DistutilsExecError as msg:
              raise LinkError(msg)
        else:
              log.debug("skipping %s (up-to-date)", output_filename)

Which may work even if the deployment model will frequently be altered by Microsoft from versions to versions ;-)

Visual Studio 2010 (Win32/Win64) packages are now available

2010-05-02T21:05:00.000+02:00

After some investigation of a couple of issues, I've finished to set up daily build packages compiled with Visual Studio 2010. Just as for the previous packages I've compiled all the dependecies with Visual Studio 2010 to prevent from potential memory corruption issues as described in my earlier post: Redistribution of the shared C runtime component in Visual C++

The packages can be downoaded from the usual location: http://vbkto.dyndns.org/sdk

In addition to the binary packages the build SDKs for VS2010 are also available (containing the libs and headers)

Although the builds seem to be in good condition (the auto tests are running correctly) don't hesitate to contact me if any further issues encountered.

Note on the MrSID driver: Currently there are no available VS2010 versions of the LizardTech GeoExpress Raster DSDK, therefore the MrSID plugin is not included in the Visual Studio 2010 packages. However you might want to copy (bin\lti_dsdk_dll.dll and bin\gdal\plugins\gdal_MrSID.dll) from the corresponding VS2008 packages in order to have MrSID support switched on.

MapServer 5.6.2 released

2010-03-20T23:39:00.000+01:00

After fixing a few important issues found in 5.6.0 and 5.6.1 the MapServer team has announced the release of MapServer version 5.6.2.
Please refer to the history page for the related changes.
The binary distributions listed in the download page should be updated with the new release in the next few days.

Changes in the MapServer MSSQL2008 driver

2010-03-06T23:01:00.000+01:00

According to the feedback of the users this driver has been broken with regards to the query handling. By looking into the details I've fixed a couple of issues, like:

- Added a default implementation of LayerResultsGetShape? (causing a systematic failure as of the single-pass query implementation)

- Changed the query shape to POINT instead of POLYGON when the area of the query shape is zero (causing a failure at the server).

- Fixed the buffer length when creating the query shape (to prevent from the memory corruption error).

- Added advanced error reporting (ODBC errors) on the msDebug output.

- Added the OID to the field list of the query select statement (causing errors when retrieving the columns).

These changes along with fixes for several further issues will are available to download from http://vbkto.dyndns.org/sdk/

Any feedback and further testing efforts would be helpful, before we issue the next stable release, MapServer 5.6.2 (hopefully in the next week)

Redistribution of the shared C runtime component in Visual C++

2010-02-21T13:48:00.001+01:00

There have been a couple of issues related to the CRT deployment with the MapServer and GDAL binary packages causing the application to fail to run properly, so you might have to keep a couple of things in mind when deploying your application on the target computer. Most of these issues are documented in the related MS articles so refer to these documents for the details.

1. Mixing dll-s compiled with different compilers/ CRT dependencies

This scenario is not supported by Microsoft and you should avoid this according to the following article: Potential Errors Passing CRT Objects Across DLL Boundaries
"When you pass C Run-time (CRT) objects such as file handles, locales, and environment variables into or out of a DLL (function calls across the DLL boundary), unexpected behavior can occur if the DLL, as well as the files calling into the DLL, use different copies of the CRT libraries.
A related problem can occur when you allocate memory (either explicitly with new or malloc, or implicitly with strdup, strstreambuf::str, and so on) and then pass a pointer across a DLL boundary to be freed. This can cause a memory access violation or heap corruption if the DLL and its users use different copies of the CRT libraries." This issue not necessarily cause problems when each dll takes the responsibility to de-allocate the memory it has been allocated within the same dll or executable. However we cannot be certain about this for each dlls deployed along with the MapServer and GDAL binary packages, but we should keep this danger out of the door as the packages have been compiled the with the same compiler and CRT setting (/MD). Just make sure you won't mix the dll-s from different packages within your deployment.

2. Expected locations of the CRT redistributables

"When you build an application in Microsoft Visual Studio, and the application uses the C run-time libraries (CRT), distribute the appropriate CRT DLL from the following list with your application:

Msvcr100.dll/Msvcp100.dll for Microsoft Visual C++ 2010
Msvcr90.dll/Msvcp90.dll for Microsoft Visual C++ 2008
Msvcr80.dll/Msvcp80.dll for Microsoft Visual C++ 2005
Msvcr71.dll/Msvcp71.dll for Microsoft Visual C++ .NET 2003 with the Microsoft .NET Framework 1.1

For Microsoft Visual C++ .NET 2003, you should install the CRT DLLs into your application program files directory. You should not install these files into the Windows system directories. For Msvcr80.dll and for Msvcr90.dll, you should install the CRT as Windows side-by-side assemblies." For more information see KB326922 and Redistributing Visual C++ Files. According to the suggestions above I've copied the msvcrt71 dll-s in the packages compiled with MSVC2003. With regards to the MSVC2005/2008 packages you should probably install the corresponding Visual C++ redistributable (vcredist_) package (or incorporate the msm module in your application installer). Make sure to install the package for your desired platform/architecture (x86 or x64)

Microsoft Visual C++ 2005 Redistributable Package (x86)

Microsoft Visual C++ 2005 Redistributable Package (x64)

Microsoft Visual C++ 2008 Redistributable Package (x86)

Microsoft Visual C++ 2008 Redistributable Package (x64)

3. Application manifests

Upon establishing the option of the side-by-side assembly cache (as of Win2003 Server and Windows XP) the binaries (dll-s or executables) should declare their (CRT) dependencies in the application manifest file. Without this declaration the application is a candidate to have the following error: C Run-Time Error R6034. "An application manifest is an XML file that describes and identifies the shared and private side-by-side assemblies that an application should bind to at run time. These should be the same assembly versions that were used to test the application. Application manifests may also describe metadata for files that are private to the application. Application manifests should be included as a resource in the application's EXE file or DLL." Fortunately the dlls and executables in the MapServer and GDAL binary packages contain these manifests embedded as a resource in the binary files (for MSVC2005/2008). This is not an issue with the packages build with MSVC2003 where the CRT implementation is located is the application directory (see #2 above) and not in the shared assembly cache.
Note: For some reason with Visual Studio 2010 Microsoft has reverted the behaviour to VS2003 and no manifests are generated by default. In these cases the CRT dll-s should be installed along with the applications.

4. Licensing issues

In general you should have a valid license for a "VC.NET 2003" product to be able to redistribute msvcr71.dll, please refer to the redist.txt file in your "Visual" product installation folder which files are allowed to be redistributed. This is not the case with the vcredist_ packages (for VS2005/2008) which are allowed to be installed on a per-user basis within the terms of the corresponding EULA.

libcurl security vulnerability is fixed in the recent release (7.20.0)

2010-02-09T23:42:00.003+01:00

libcurl 7.20.0 is released by fixing a vulnerability described in Security Advisory February 9 2010 . Maintainers of the binary distributions are suggested to take one of the following actions as soon as possible:

A - Upgrade to curl and libcurl 7.20.0
B - Apply this patch and rebuild
C - Disable automatic content encoding decompression in your application
D - Rebuild curl without zlib support
E - change your code to use 4*CURL_MAX_WRITE_SIZE for buffer sizes

GDAL 1.7 latest fixes are available for testing

2010-02-06T23:01:00.003+01:00

The GDAL 1.7 branch have now been added to the Windows builders providing ready to use daily stapshots of the Windows binaries for being up-to-date with all the recent changes. These packages should contain the fixes in GDAL since the latest release of 1.7.0 (2010/01/19) including the fix for the issue with the HFA driver which is considered as a blocker, and validates a new GDAL release (version 1.7.1) within a couple of days.

Welcome!

2010-01-31T00:00:00.005+01:00

I've started this blog to provide futher information about my recent activities and thinking in Open Source GIS.

My name is Tamas Szekeres, I am an open source geospatial software developer from Hungary who has been working on several GIS related projects up to the present. I'm a long term contributor / project steering committe member / C# interface maintainer of the GDAL and MapServer projects for years.

Currently I provide consultancy services related to GIS which involves to develop custom desktop and web mapping applications for Linux/Windows and Windows Mobile platforms in C/C#.

In order to provide further help in GIS development I maintain regular builders to provide daily built ready to use windows (Win32/Win64) binary packages and software development kits to make compling GDAL and MapServer as easy as pie. You can always obtain the cutting edge versions of MapServer and GDAL containing all the recent fixes made in the stable and development versions.

I'm the maintainer of several automated builders for GDAL and MapServer in order to make the development easier for the contributors as well. These build-slaves are all based on the SDKs mentioned above.

If you are interested in my recent news in GIS don't miss to follow my further posts.

Best regards,

Tamas