Données ouvertes et les émissions de Gaz à effet de serre au Canada

#cop17 ça vous dit quelque chose?  On dénombre pas moins de 60 000 tweets contenant cette mention depuis une trentaine de jours  sur twitter!  Je ne ferai pas ici de la petite propagande climatologique rassurez-vous, par contre c'était bien assez pour piquer ma curiosité ! Trouvant le sujet de cette conférence sur le changement climatique plutôt intéressant d'un point de vue professionnel et personnel, j'ai donc fouillé un peu le sujet pour visualiser la production de  Gaz à effet de serre (GES) au Canada.

En 2008 selon "The World Ressource Institute", le Canada se situait au 8e rang dans l'échelle des plus grands émetteurs de GES avec 561 MtCO2e soit 1.87 % de la production mondiale totale (toutes sources GES confondues).  Globalement c'est peu mais certains diront que c'est déjà trop alors pour voir plus précisément comment se distribuait la production de dioxyde de carbone au Canada j'ai fait quelques recherches sur le Portail de données ouvertes du gouvernement du Canada.  Ce site regorge de données et il suffit de se donner la peine de chercher un peu pour trouver tout plein de données à référence spatiale.  J'ai trouvé des données et une application de cartographie interactive assez rapidement en recherchant le mot clé "Gaz".

Avec ces données téléchargées (format CSV) sur le site d'Environnement Canada, ce que je voulais faire était assez simple: localiser un peu mieux sur une carte la production des GES au Canada.  J'ai géoréférencé les 578 installations émettrices de GES recensées au Canada dans QGis et j'ai ensuite créé des buffers autour des installations pour visualiser leur importance au Canada et un aussi produit grid qui pourra faciliter la représentation cartographique de ces installations.  Parce que je ne voulais faire autre chose qu'une "Heat map", après quelques tentatives,  j'arrive à quelque chose d'intéressant visuellement à l'échelle du Canada (point rouge étant une installation industrielle émettrice de GES et buffer de 100, 250 et 500 km de rayon).

 

Pour produire les valeurs permettant de créer une représentation thématique du sujet (et surtout me faciliter la vie dans mon cas) je transfère les installations et mon grid hexagonal dans Postgresql/Postgis.  J'ai fait quelques requêtes pour comptabiliser 'spatialement' les données.

1) petit nettoyage des données
update hex_emis_ges set "Emissions"='0' where "Emissions"='-99.00000000' or "Emissions" is null
update hex_emis_ges set "Emission_2"='0' where "Emission_2"='-99.00000000' or "Emission_2" is null
update hex_emis_ges set "Emissio3"='0' where "Emissio3"='-99.00000000' or "Emissio3" is null
update hex_emis_ges set emis_ge3_3=to_number("Emissions",'999999') + to_number("Emission_2",'999999')+to_number("Emissio3",'999999');
2) J'ai ensuite simplement représenté l'importance du nombre d'installations émettrices en GES en créant des tables buffers de 100, 250 et 500 km autour des installations.
create table buf500km_install_ges as select "Emissions" as emissions,ST_Buffer(install_ges_p.the_geom,500000,'quad_segs=8') as the_geom from install_ges_p;
CREATE INDEX idx_buf500km_install_ges_geom_gist ON buf500km_install_ges USING gist(the_geom);
create table buf250km_install_ges as select "Emissions" as emissions,ST_Buffer(install_ges_p.the_geom,250000,'quad_segs=8') as the_geom from install_ges_p;
CREATE INDEX idx_buf250km_install_ges_geom_gist ON buf250km_install_ges USING gist(the_geom);
create table buf100km_install_ges as select "Emissions" as emissions,ST_Buffer(install_ges_p.the_geom,100000,'quad_segs=8') as the_geom from install_ges_p;
CREATE INDEX idx_buf100km_install_ges_geom_gist ON buf100km_install_ges USING gist(the_geom);
3) Enfin, j'ai comptabilisé le nombre d'intersections des buffers sur mon grid hexagonal
CREATE TABLE hex_emis_ges_cnt_install_tt as
      select hex_emis_ges.gid, "Latitude", "Longitude" ,"Emissions","Emission_2","Emissio3",
             (select count(*) from buf500km_install_ges where ST_Intersects (buf500km_install_ges.the_geom,
                      hex_emis_ges.the_geom) = true) as nb_buf500,
             (select count(*) from buf250km_install_ges where ST_Intersects(buf250km_install_ges.the_geom,
                      hex_emis_ges.the_geom)=true) as nb_buf250,
             (select count(*) from buf100km_install_ges where ST_Intersects(buf100km_install_ges.the_geom,
                      hex_emis_ges.the_geom)=true) as nb_buf100,the_geom from hex_emis_ges;
CREATE TABLE hex_emis_ges_cnt_install as
select hex_emis_ges_cnt_install_tt.*, (nb_buf500+nb_buf250)+nb_buf100 as nb_tot_install from hex_emis_ges_cnt_install_tt;
drop table hex_emis_ges_cnt_install_tt;
CREATE INDEX idx_hex_emis_ges_cnt_install_geom_gist ON hex_emis_ges_cnt_install USING gist(the_geom);

Le résultat n'est certes pas une révolution ni parfaite et encore moins scientifique, mais l'effet visuel me semble intéressant dans QGis.  La carte montre deux choses:

  • La quantité d'émission de GES par installations industrielles recensées au Canada
  • La concentration et la répartition spatial des installations industrielles au Canada
  Je n'aimais pas vraiment le résultat alors j'ai passé le projet dans la moulinette TileMill pour produire une carte un peu plus soignée.  Il ne me restera qu'à placer la carte tuilée (MBtiles) dans une page pour présenter l'ensemble du territoire et que j'aimerais bien faire avec MapCache cette fois... Prochain billet donc à suivre!  

Autres liens:

http://cait.wri.org/cait.php?page=background&from=yearly http://www.carbon-biodiversity.net/Interactive/CarbonCalculatorNotes http://maps-cartes.ec.gc.ca/indicators-indicateurs/TableView.aspx?ID=15 http://www.protectedplanet.net/  

Nouveaux OUTPUTFORMAT de Mapserver 6.0

Une des nouveautés de Mapserver 6.0 est l’ajout de nouveaux formats de sortie liés aux requêtes GetFeature via WFS. Attention par contre, on doit obligatoirement installer GDAL/OGR 1.8. Dans mon cas test sur mon serveur Debian, je devais utiliser une connexion Oracle. J'ai donc compilé GDAL/OGR avec le support Oracle et compilé Mapserver pour lire ce format via GDAL/OGR. J'ai aussi préparé un fichier regroupant les formats de sortie suivant: OUTPUTFORMAT   NAME "SHAPEZIP"   DRIVER "OGR/ESRI Shapefile"   MIMETYPE "application/shapefile"   FORMATOPTION "STORAGE=filesystem"   FORMATOPTION "FORM=zip"   FORMATOPTION "FILENAME=result.zip" END OUTPUTFORMAT   NAME "MIDMIF"   DRIVER "OGR/MapInfo File"   FORMATOPTION "STORAGE=filesystem"   FORMATOPTION "FORM=multipart"   FORMATOPTION "DSCO:FORMAT=MIF"   FORMATOPTION "FILENAME=result.mif" END OUTPUTFORMAT   NAME "MultiMIDMIF"   DRIVER "OGR/MapInfo File"   FORMATOPTION "STORAGE=filesystem"   FORMATOPTION "FORM=multipart"   FORMATOPTION "DSCO:FORMAT=MIF"   FORMATOPTION "FILENAME=result" END OUTPUTFORMAT   NAME "CSV"   DRIVER "OGR/CSV"   MIMETYPE "text/csv"   FORMATOPTION "LCO:GEOMETRY=AS_WKT"   FORMATOPTION "STORAGE=filesystem"   FORMATOPTION "FORM=simple"   FORMATOPTION "FILENAME=result.csv" END OUTPUTFORMAT   NAME "CSVSTREAM"   DRIVER "OGR/CSV"   MIMETYPE "text/csv; streamed"   FORMATOPTION "LCO:GEOMETRY=AS_WKT"   FORMATOPTION "STORAGE=stream"   FORMATOPTION "FORM=simple"   FORMATOPTION "FILENAME=result.csv" #-- If “stream” then the datasource will be created with a name “/vsistdout” as an attempt to write directly to stdout. Only a few OGR drivers will work properly in this mode (ie. CSV, perhaps kml, gml) END OUTPUTFORMAT   NAME "OGRGML"   DRIVER "OGR/GML"   MIMETYPE "text/xml; subtype=gml/2.1.2; driver=ogr"   FORMATOPTION "STORAGE=memory"   FORMATOPTION "FORM=multipart"   FORMATOPTION "FILENAME=result.gml" END OUTPUTFORMAT   NAME kml   DRIVER "KML"   MIMETYPE "application/vnd.google-earth.kml+xml"   IMAGEMODE RGB   EXTENSION "kml"   FORMATOPTION 'ATTACHMENT=result.kml'   FORMATOPTION "maxfeaturestodraw=100" END OUTPUTFORMAT   NAME kmz   DRIVER "KMZ"   MIMETYPE "application/vnd.google-earth.kmz"   IMAGEMODE RGB   EXTENSION "kmz"   FORMATOPTION 'ATTACHMENT=result.kmz' END OUTPUTFORMAT   NAME "geojson"   DRIVER "TEMPLATE"   MIMETYPE "application/json; subtype=geojson"   FORMATOPTION "FILE=../template/geojson.txml" END OUTPUTFORMAT   NAME "customxml"   DRIVER "TEMPLATE"   FORMATOPTION "FILE=mon_template.xml" END J'ai testé avec ce Mapfile WFS plusieurs de ces formats. Pour le type “SHAPEZIP” par exemple, on fait simplement cette requête URL: http://sigtest.gouc.qc/cgi-bin/extract?SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&typename=SMDTS_TERR&OUTPUTFORMAT=SHAPEZIP NOTE 1: Je n'ai pas ajouter de clause FILTER pour mon test mais c'est possible. Voir ici pour plus d'explications NOTE 2: J'ai n'ai pas fait d'essais avec le format de sortie TEMPLATE qui nous permet de personnaliser un format de sortie texte comme geojson par exemple. Cette option est très puissante et mérite d'être mieux connue. Imaginer utiliser Mapserver pour générer du code javascript pour jQuery par exemple...! NOTE 3: Par comparaison, avec Geoserver qui support déjà ce type de format de sortie (SHAPEFILES) depuis plus longtemps: http://sigtest.gouv.qc:8180/geoserver/wfs?SERVICE=WFS&VERSION=1.0.0&REQUEST=GetFeature&typename=SMDTS_TERR&OUTPUTFORMAT=SHAPE-ZIP

Database-driven mapfile

Avez-vous déjà essayé de construire une mapfile qui serait dynamiquement contrôlé par la base de données? À l'occasion, cette astuce peut aider à produire des services Web cartographiques basés sur des données dynamiques d'un système d'envergure. Voici une description de table provenant de Postgesql: Table "public.om_users" Column      | Type                   | Modifiers ------------+------------------------+------------------------------------- om_id       | integer                | not null longitude   | numeric(12,9)          | latitude    | numeric(12,9)          | color       | character varying(11)  | default '255 0 0'::character varying label       | character varying(100) | the_geom    | geometry               | feature     | character varying(32)  | default 'circle'::character varying size        | integer                | default 5 outline     | character varying(11)  | default '0 0 0'::character varying Indexes: "om_users_pkey" PRIMARY KEY, btree (om_id) Check constraints: "enforce_dims_the_geom" CHECK (st_ndims(the_geom) = 2) "enforce_geotype_the_geom" CHECK (geometrytype(the_geom) = 'POINT'::text OR the_geom IS NULL) "enforce_srid_the_geom" CHECK (st_srid(the_geom) = 4326) On place simplement les champs de données au bon endroit dans le mapfile. Pour chacun des enregistrements, mapserver prendra les données de la table pour cartographier l'entité. LAYER   NAME "UsersLayers"   INCLUDE "ec/ec_connec_db_pg.map"   DATA "the_geom from om_users using srid=4326"   TYPE POINT   METADATA     "wms_name" "UsersLayer"     "wms_title" "UsersLayer"     "wms_server_version" "1.1.1"     "wms_srs" "EPSG:4326 EPSG:900913"     "gml_featureid" "oid"     "gml_include_items" "all"   END   PROJECTION     "init=epsg:4326"   END   CLASSITEM "om_id"   CLASS   NAME "Status"   STYLE     SYMBOL [feature]       COLOR [color]       OUTLINECOLOR [outline]       SIZE [size]     END   END END

A not well documented Mapserver feature for Hillshade

If you have a hillshade GRID(or exported tiff) and DEM you can build a nice color hillshape map with Mapserver...  It's not really simple and well documented, but it's possible. I think this trick was add in Mapserver in 'experimental' mode in the trunk and gets released, but never realy discuss in the mailling list Mapserver-dev. To build this kind of service, we must have a DEM and Hillshade. It's not necessary to have a GRID format for this trick. A simple black and white tif format is fine. First you must know the pixel range to process in your DEM. You need it to let GDAL reader a pre-scale from incoming raster data to get the pixel range to process. You can easely use the gdalinfo utilities. ... Upper Left (-1051654.551, 2174097.741) ( 88d52'54.70"W, 62d14'24.12"N) Lower Left (-1051654.551, -201102.259) ( 80d59'1.05"W, 41d23'0.63"N) Upper Right ( 1359945.449, 2174097.741) ( 42d36'0.34"W, 61d21'52.09"N) Lower Right ( 1359945.449, -201102.259) ( 52d27'53.73"W, 40d50'27.53"N) Center ( 154145.449, 986497.741) ( 66d11'34.47"W, 52d52'40.66"N) Band 1 Block=256x256 Type=Int16, ColorInterp=Gray Min=-10.000 Max=706.000 Minimum=-32768.000, Maximum=1890.000, Mean=-185.108, StdDev=3252.745 ... In the example I have in Band 1, a min elevation of -10 to a max of 706. This resulte in the mapfile with PROCESSING "SCALE=-10,706". Then, you have to process the DEM dynamically with a COLORRANGE in TRANSPARENCY mode and overlay it on a hillshade. To get the DEM and the process hillshade in a single layer in your map, simply GROUP them in your mapfile. Example in the mapfile GROUP on "Ombrage_bleu_250K"...

...
LAYER
    NAME "ombre250k_blue"
    GROUP "Ombrage_blue_250K"
    DATA "mne250k/ombre250k.tif"
    TYPE RASTER
    MAXSCALE 4000000
    MINSCALE 1
    METADATA
        "wms_group_title"       "Ombrage blue 250K"
        "wms_name"              "ombre250k_blue"
        "wms_title"             "ombre250k blue"
        INCLUDE                 "../include/ec/ec_meta_layer.map"
    END
    PROJECTION
         "init=epsg:32198"
    END
END
LAYER
    NAME "mne250k_blue"
    GROUP "Ombrage_blue_250K"
    DATA "mne250k/mne250k.tif"
    TRANSPARENCY 30
    TYPE RASTER
    PROCESSING "SCALE=-1,706"
    METADATA
        "wms_group_title"       "Ombrage bblue 250K"
        "wms_name"              "mne250k_blue"
        "wms_title"             "mne250k blue"
        INCLUDE                 "../include/ec/ec_meta_layer.map"
    END
    PROJECTION
        "init=epsg:32198"
    END
    CLASS
        STYLE
            COLORRANGE 255 255 255   0 0 255
            DATARANGE -1 706
        END
    END
END
...
End the getMap query, use the group layer name "Ombrage_bleu_250K"... http://your.server.com/cgi-bin/mapserv?map=demshade&REQUEST=GetMap&SERVICE=WMS&VERSION=1.1.1&LAYERS=Ombrage_bleu_250K&STYLES=&FORMAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&SRS=EPSG:32198&BBOX=87557.4046920096,498799.484254657,182346.544021857,583840.349445035&WIDTH=739&HEIGHT=663 In next example, I have tree layer that I can call with the GROUP tag "HILLSHAPE_GREEN_BLUE". First I add the hillshape in background. Then I overlay two MNE rasters layers that show elevation from -63 to 1280. So I coloring for the first -63 to 500 metre elevation with a green to blue colo rampe, and from 300 to 1280 with a blue to green color rampe. Not that all thos layer are add in the mapfile as Arc/INFO GRID TILEINDEX.

LAYER
    NAME  "Ombre_vert_bleu"
    GROUP  "HILLSHAPE_GREEN_BLUE"
    TYPE  RASTER
    TILEINDEX  "mneomb20k/omb"
    TILEITEM  "location"
    MAXSCALE  200000
    MINSCALE  1
    METADATA
        "wms_group_title"    "Ombrage  vert  bleu  20K"
        "wms_name"           "Ombre_vert_bleu"
        INCLUDE              "../include/ec/ec_meta_layer.map"
        "wms_title"          "Ombre"
    END
    PROJECTION
        "init=epsg:32198"
    END
END
  #-------------------------
LAYER
    NAME  "MNE_comp_green_blue"
    GROUP  "HILLSHAPE_GREEN_BLUE"
    TILEINDEX  "mneomb20k/mne"
    TILEITEM  "location"
    TRANSPARENCY  40
    TYPE  RASTER
    PROCESSING  "SCALE=-63,500"
    MAXSCALE  200000
    MINSCALE  1
    METADATA
        "wms_name"         "MNE_comp_green_blue"
        INCLUDE            "../include/ec/ec_meta_layer.map"
        "wms_title"        "MNE  comp  green"
    END
    PROJECTION
            "init=epsg:32198"
    END
    CLASS
        STYLE
            COLORRANGE  34  139  34  255  255  255
            DATARANGE  -63  500
        END
    END
END
#--
LAYER
    NAME  "MNE_comp_blue_green"
    GROUP  "HILLSHAPE_GREEN_BLUE"
    TILEINDEX  "mneomb20k/mne"
    TILEITEM  "location"
    TRANSPARENCY  40
    TYPE  RASTER
    PROCESSING  "SCALE=300,1280"
    MAXSCALE  200000
    MINSCALE  1
    METADATA
        "wms_name"      "MNE_comp_blue_green"
        INCLUDE         "../include/ec/ec_meta_layer.map"
        "wms_title"     "MNE  comp  bleu"
    END
    PROJECTION
        "init=epsg:32198"
    END
    CLASS
        STYLE
            COLORRANGE  255  255  255  0  0  205
            DATARANGE  300  1280
        END
    END
END

The result of this combination show here Hillshade green-blue

New Spatialite format with Mapserver and GDAL/OGR 1.7.0

Sometime we find something such simple as stupide ... I think that Spatialite is one of that.  If you look for a GIS storage format AND tabular data, very simple, open, easy to use and manage with Mapserver, python or php, take a look to this solution. Note that with a single database file you can build, query and manage your data like MySql or Postgresql. For my test, I use the open data Natural Earth, and it work at my first try ..! :) Rock On!! My NaturalEarth.sqlite file can be dowload here. (Natural Earth. Free vector and raster map data @ naturalearthdata.com). I load it with the Spatiallite-gui tools and than visualize them with the Spatiallite-gis... It take's me 15 minutes... Including the download and install steps... To make my first WMS Spatialite data test, I build a mapfile for Mapserver with this cool free vector dataset of Natural Earth. I just have to replace the shapefiles connection string with a OGR connection type and specify the table name:         CONNECTIONTYPE OGR         CONNECTION "path_to_spatialite_file"         DATA "50m-geography-marine-polys" Also, note that you HAVE to use the latest GDAL/OGR(version 1.7.0). If you use MS4W, you can download the beta10 or later. My first impression is that is not as fast as shapefile format. I gona test for larger dataset later... But, according to OGR specification web page, this driver still don't take advantage of spatial index?. So it's not a negative tips of this format... My mapfile test is here: MAP     NAME "spatialite"     EXTENT -180 -90 180 90       SIZE 700 500     IMAGETYPE PNG     IMAGECOLOR 0 0 0       UNITS METERS     SYMBOLSET                   "/ms4w/msp/symbols/commun/symbols.map"     FONTSET                     "/ms4w/msp/fonts/commun/fonts.txt"     CONFIG MS_ERRORFILE         "/ms4w/tmp/mapserv.log"     WEB         QUERYFORMAT     "text/xml"         BROWSEFORMAT    "text/xml"         IMAGEPATH       "/srv/www/msp/services/tmp/"         IMAGEURL        "/ms_tmp/"             METADATA            "wms_title"                  "spatialite test"            "wms_name"                   "spatialite"            "wms_abstract"               ""            "wms_description"            ""            "wms_keywordlist"            "Spatialite test"            "wms_onlineresource"         "http://localhost/cgi-bin/mapserv.exe?map=C:/Travail/spatialite/spatialite.map"         END     END     PROJECTION         "init=epsg:4326"     END     LAYER         NAME "50m-admin-1-states-provinces-shp"         DATA "C:/Travail/spatialite/NaturalEarth/50m-admin-1-states-provinces-shp.shp"          TYPE polygon         PROJECTION             "init=epsg:4326"         END         METADATA             "wms_name"              "50m-admin-1-states-provinces-shp"             "wms_title"             "states-provinces shapefiles"             "wms_keywordlist"       ""             "wms_server_version"    "1.1.1"         END         CLASS             NAME "states-provinces"             STYLE                 COLOR 255 90 90             END         END     END         LAYER         NAME "50m-admin-0-countries"         CONNECTIONTYPE OGR         CONNECTION "C:/Travail/spatialite/NaturalEarth/NaturalEarth.sqlite"  # full path to SQLite db file         DATA "50m-admin-0-countries"         TYPE polygon         PROJECTION             "init=epsg:4326"         END         METADATA             "wms_name"              "50m-admin-0-countries"             "wms_title"             "50m-admin-0-countries"             "wms_keywordlist"       ""             "wms_server_version"    "1.1.1"             "wms_extent"            "-180 -90 180 90"         END         CLASS             NAME "states-provinces"             STYLE                 COLOR 255 190 190             END         END     END      LAYER         NAME "50m-geography-marine-polys"         CONNECTIONTYPE OGR         CONNECTION "C:/Travail/spatialite/NaturalEarth/NaturalEarth.sqlite"  # full path to SQLite db file         DATA "50m-geography-marine-polys"          TYPE polygon         PROJECTION             "init=epsg:4326"         END         METADATA             "wms_name"              "50m-geography-marine-polys"             "wms_title"             "50m-geography-marine-polys"             "wms_keywordlist"       ""             "wms_server_version"    "1.1.1"             "wms_extent"            "-180 -90 180 90"         END         CLASS             NAME "states-provinces"             STYLE                 COLOR 24 116 205             END         END     END  END 

A new XML Mapfile Format in Mapserver 5.6

Build a mapfile for Mapserver from plain text is not an easy job.  We can use a Syntax coloring Editor like SciTE, PsPad or UltraEdit, but we all hope have a nice and easy  mapfile editor for build our web map service.  Sice version 5.6, Mapserver has a XML schema has been defined to encode mapfiles in XML format. We can get some input from Mapserver Wiki for example, how to implement it and how to convert existing mapfile. <?xml version="1.0" encoding="UTF-8"?> <Map name="GMAP-DEMO" version="5.6.0" status="ON" xmlns="http://www.mapserver.org/mapserver" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mapserver.org/mapserver"> <extent>-2200000 -712631 3072800 3840000</extent> <fontSet>../etc/fonts.txt</fontSet> <imageColor red="255" green="255" blue="255"/> <Layer name="bathymetry" type="RASTER" status="ON"> <data>bath_mapserver.tif</data> <Metadata> <item name="DESCRIPTION">Elevation/Bathymetry</item> </Metadata> </Layer> <Layer name="popplace" type="POINT" status="ON"> <Class name="Cities"> <color red="0" green="0" blue="0"/>< expression>1</expression> <Label type="TRUETYPE"> <align>LEFT</align> <color red="255" green="0" blue="0"/> <font>sans-italic</font> <outlineColor red="255" green="255" blue="255"/> <partials>FALSE</partials> <position>AUTO</position> <size>8</size> </Label> ...So this feature is not realy exciting BUT now we have a tool to parse any mapfile and option to build new client interface for Mapfile.  This is a prety good news... :-)

How to produce 8bits AGG homemade colors palette with Mapserver

The format of the output image affects the quality and size of the image to be transferred to the client on the web. We are always trying to minimize the size without losing image quality. With Mapserver, we can use the format AGG (24bit or 32bit PNG) of high quality although it's also a format that produces larger image size(in Kb). But, it's possible to force Mapserver to produce images of excellent qualities AGG 8bit format. The default image format output in Mapserver is 8bit PNG format (256 colors). To test a 24bits or 32bits format, you must overwrite the default format in your mapfile with this in your MAP tag. OUTPUTFORMAT    NAME "png"    MIMETYPE "image/png"    DRIVER "GD/PNG"    EXTENSION "png"    #--IMAGEMODE RGB #-- Driver PNG 24bit    IMAGEMODE RGBA #-- Driver PNG 32bit    TRANSPARENT ON END The default AGG format in Mapserver is a 24bits driver but you can use a 8bits driver. This type of driver in your mapfile will cause a slower performence of Mapserver. You can specify a predefined colors palette to improved performance and get accurate colors of your map. To build your homemade colors palette, you have first to produce a 8bits AGG image by mapserver with this driver in your mapfile: OUTPUTFORMAT    NAME "png8bitaggauto"    DRIVER "AGG/PNG"    MIMETYPE "image/png"    IMAGEMODE "pc256"    EXTENSION "png"    FORMATOPTION "TRANSPARENT=ON"    FORMATOPTION "INTERLACE=OFF"    FORMATOPTION "QUANTIZE_FORCE=ON"    FORMATOPTION "QUANTIZE_COLORS=256" END I suggest to produce more than just one image. Get images from multiple scales and different sectors of your WMS to get every colors. After, build a mosaic all images with an imaging tool like Microsoft Paint or Gymp. Finally, use gdalinfo to get your palette colors and paste them in a ASCII file. To use your new palette colors and realy improve your Mapserver AGG image output, add this driver specification in your mapfile(in MAP tag) #-- Driver AGG 8bits OUTPUTFORMAT    NAME "png8bitsagg"    DRIVER "AGG/PNG"    MIMETYPE "image/png"    IMAGEMODE "rgba"    EXTENSION "png"    FORMATOPTION "TRANSPARENT=ON"    FORMATOPTION "PALETTE_FORCE=TRUE"    FORMATOPTION "PALETTE=E:ms4wpalpal_agg256.txt"    FORMATOPTION "INTERLACE=OFF" END

How to have nice symbol pixmap with transparency in Mapserver

First, to have a nice symbol pixmap, you should reduce the size of the image the same size it should appear in the map. To do this, you must not resize the symbol in the class layer of your mapfile.  It may be necessary to resample your image. Use Gimp(a photoshop Open sources) for this. For transparency works well, use a gif format. If your symbol(image) does not use transparency, you can quickly create one with Microsoft Photo Editor.  You only need to identify the background color with Transparent Color function.  This little function allows you to quickly create a NoData Value for image. Then, you must know the index(NoData Value) of transparent color of your symbole. This is not the transparent RGB color. To do this, you can use the utility gdalinfo to get the transparent index color. In this example, we have NoData Value = 254. Finally, just have to put this value in the symbol pixmap of your mapfile: SYMBOL    NAME "nn"    TYPE PIXMAP    IMAGE "1nn.gif"    TRANSPARENT 254 END

Optimizing your mapfile for Mapserver

There are many tricks to optimize your mapfile.  After many years to build mapfiles for Mapserver, I learned many trick to optimise a mapfile.  I try here to highlight some of them:
  1. Manage your EPSG file if you specify espg code.  Only keep in your file projections used in your systems.  The original file have approx 545k and Mapserver read this file each time you add in your mapfile instruction like  "init=epsg:32198".  You can reduce it to 4k. Or, you can put the projections specify in your mapfile at the top of your epsg file...  so Mapserver don't need to scan the whole epsg file.
  2. Lowercase epsg intruction like "init=epsg:32198".  Mapserver use C function "strcmp" in lowercase first.
  3. Try to be "brief"... Don't put extra default instruction like STATUS ON or LABELCACHE ON... STATUS and LABELCACHE are ON by default.  The less you have to read, faster you are. And try to limit your comment to...
  4. In the layer definition the tag EXPRESSION can be very costly.  Alway try to use "regular expression"(REGEX) it's much faster!  If you add something like that EXPRESSION (("[DESCRIPTION]" eq "Park") OR ("[DESCRIPTION]" eq "Base") Mapserver gona built sql string then gona filtre your data! It work but it's not optimize.  In this example we just put directly in the mapfile EXPRESSION /^Park$|^Base$/ and Mapserver use it without any string manipulation.  And change this EXPRESSION ("[DESCRIPTION]" eq "Park") to this EXPRESSION "Park"!
  5. Try not to re-project your data it's costly overhead.
  6. Shapefile format ALWAYS de faster way to publish data.
  7. Shptree each of your shapefile AND don't specify shp in DATA tag.  This one is very peculiar one! If you put DATA like this DATA "/srv/data/park.shp" Mapserver use a built in reader and don't use index qix file.  But if you type DATA "/srv/data/park" Mapserver gona use OGR to read data and OGR alway use qix file!
  8. For large shapefiles, use tile4ms program to tile them.  Follow those instruction to split your shapefile.  This trick is very very usefull
  9. If you use PostGIS or Oracle Data, try to connect to only one server and keep your connection open with this instruction in each layer PROCESSING "CLOSE_CONNECTION=DEFER"
  10. Never use SIZEUNITS in the LAYER tag.  Put it in the MAP tag.
  11. If you build a mapfile for a Web map service(WMS) always add STATUS OFF in each layer.  In this way Mapserver don't prepared all cartographics parametres like labeling engine.
  12. Finally, use debug option to find out witch layers are to slow.  The best trick her is to use shp2img program to eliminate overhead of your network.  To use debug option you have to insert in the MAP tag this CONFIG MS_ERRORFILE "/srv/log/debug.log" and in each layer this tag DEBUG 5  aIn the log file check the msDrawMap for each layer.
Shp2imp example on Windows shp2img -m C:Testimg2.map -l png -o C:Testext.png -e 219589 5347860 226045 5353101 Log example [Fri Feb 15 16:05:14 2008].742000 msDrawRasterLayerLow(tif): entering. [Fri Feb 15 16:05:14 2008].762000 msDrawGDAL(): src=0,0,33477,25307, dst=0,26,935,707 [Fri Feb 15 16:05:14 2008].762000 msDrawGDAL(): red,green,blue,alpha bands = 1,2,3,0 [Fri Feb 15 16:05:15 2008].2000 msDrawMap(): Layer 2 (tif), 0.260sp