incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ramirez, Paul M (398J)" <>
Subject Re: [Proposal] HotdoG for the Apache Incubator
Date Tue, 28 May 2013 13:40:14 GMT

Bumping for comments since last week was a busy week with voting. I'd like
to call a vote on this proposal in 72 hours, pending no major changes need
to be made. 

--Paul Ramirez

On 5/21/13 10:55 AM, "Ramirez, Paul M (398J)"
<> wrote:

>Dear ASF members,
>We would like to propose HotdoG for the incubator. Paul Ramirez (me) will
>be the Champion for this project. The proposal draft is available at:
>Looking forward to integration your suggestions and feedback.
>= HotdoG Proposal =
>== Abstract ==
>The HotdoG project is an effort and suite of tools to convert HDF/HDF-EOS
>format into GeoTIFF format. This is the first of a potential series of
>incoming projects originating from The HDF Group -- the non-profit
>organization is interested in evaluating the ASF as a potential home for
>many of its projects. The HDF Group is an independently funded
>organization that started many years ago with major investment from NASA
>as the Hierarchical Data Format (HDF), version 4 and now version 5, is
>the ''de facto'' remote sensing data format for NASA missions, and an
>increasing number of other disciplines including bio medicine, radio
>astronomy, climate science, and other domains.
>HDF is both a data and metadata format, as well as a model for
>representing and access information. There are numerous downstream tools
>that can read and write HDF data, including a growing number of
>Geospatial data tools (ESRI-based, and also OpenGeo and other community
>led efforts). In addition, major interoperability efforts are also
>occurring between the remote sensing community and the climate modeling
>community (which has traditionally favored NetCDF as opposed to HDF)
>because of the efforts in HDF5 to leverage a common data format and model.
>HotdoG is poised to be a first of its kind in the form of bringing one of
>the major 2 data formats for science to the ASF (the other being the
>NetCDF format).
>== Proposal ==
>HotdoG is a software converter that converts Earth Science data in
>HDF/HDF-EOS format into GeoTIFF format. Doing so easily enables users of
>remote sensing data to interoperate with common GIS tools (like WebGIS,
>Web Processing, image analysis, and geo computational tools). We feel
>that the project is an incremental step, and an appropriate focus with
>tangible success possibilities by restraining our focus to HDF/HDF-EOS to
>GeoTIFF conversion.
>There are numerous interesting paths that we can take the toolkit in --
>as conversion from remotely sensed data to GeoTIFF involves ensuring that
>the HDF-EOS metadata elements can be appropriately represented using
>GeoTIFF headers, and the associated format. Furthermore, capturing the
>HDF's appropriate geo datum in GeoTIFF will be another important
>== Background ==
>GeoTIFF is a data and metadata standard for Earth science applications.
>It is based on binary Tagged Image File Format (TIFF).  A GeoTIFF file
>has geographic (or cartographic) data embedded as tags within the TIFF
>file that are used to geo-locate the image. This is required for correct
>integration of the image in Geographic Information Systems (GIS) and
>other popular tools like Google Earth Pro.
>In the recent years, GeoTIFF has gained popularity as a visualization
>format among NASA HDF Earth science user communities according to the
>NASA data user's survey. However, the conversion from HDF to GeoTIFF is
>not straightforward for end users because NASA HDF data products are
>diverse and organized in many different ways. For example, go to
> and you'll see many scripting language examples
>because no single script can correctly visualize all NASA HDF data.
>== Rationale ==
>The HEG tool is limited to some NASA HDF-EOS2 products (no support for
>HDF-EOS5 products) and it is not an open-source tool. The latest GDAL
>(version 1.9.2 and above) is an open-source tool but it cannot handle
>many non-HDF-EOS NASA HDF products such as TRMM (pure HDF4) and Aquarius
>(pure HDF5) correctly and automatically.
>== Initial Goals ==
>We'll improve GDAL to support NASA HDF products better by handling
>geo-location information and physical meaning of data correctly and
>We'll handle NASA products intelligently so novice users don't have to
>supply many options or figure out the details about the data products.
>For advanced users, we'll give a full control of accessing HDF products
>in many different ways so that the converted GeoTIFF file is
>scientifically valid and meaningful.
>We aim to provide command line tools first and evolve them into a GUI
>== Current Status ==
>We're looking for developers and sponsors.
>=== Meritocracy ===
>We will discuss the milestone and the future plan in an open forum. We
>plan to encourage an environment that supports a meritocracy. The
>contributors will have different privileges according to their
>=== Community ===
>GIS and Earth Science
>=== Core developers ===
> * H. Joe Lee <hyoklee AT hdfgroup DOT org>
> * Denis Nadeau <denis DOT nadeau AT gmail DOT com>
> * Pedro Vicente <pvicente AT uci DOt edu>
>=== Alignment ===
>HotdoG employs H4CF Conversion Toolkit for reading data and GDAL for
>writing GeoTIFF. In addition, we plan to integrate HotdoG with other
>products from new missions such as SMAP and IceSAT-2.
>If HEG becomes available as open source, HotdoG will employ it as well
>for HDF-EOS2 products.
>== Known Risks ==
>HDF-EOS2 and HDF-EOS5 libraries are distributed under the
>[[|NOSA license]] that may not be
>compatible with Apache License. This will be vetted with the Legal
>Affairs Committee and if found incompatible we will find a different way
>to read HDF-EOS products.
>HDF is a flexible format which affords for unusual objects such as Point
>& Zonal Averages (HDF-EOS), VData (HDF4), and compound datatype (HDF5)
>that present challenges in mapping to GeoTIFF. To mitigate this challenge
>the initial goals will focus on those objects which are more common place
>and easily map to GeoTIFF.
>== Documentation ==
>User's guide documentation will be provided via Doxygen. The guide will
>contain specific GeoTIFF conversion examples for each NASA HDF data
>== External Dependencies ==
> * H4CF Conversion Toolkit
> * GDAL
> * HDF4 / HDF5
>== Required Resources ==
>=== Mailing List ===
> * hotdog-private (with moderated subscription)
> * hotdog-dev
> * hotdog-commits
>=== Issue Tracking ===
>JIRA HotdoG (HotdoG)
>=== Other Resources ===
> * Wiki
>  *
>== Initial Committers ==
> * H. Joe Lee (The HDF Group)
> * Paul Ramirez (NASA JPL)
> * Chris Mattmann (NASA JPL)
> * Lewis John McGibbney (Stanford University)
> * Denis Nadeau (NASA NCCS)
> * Pedro Vicente (University of California at Irvine)
> * Babak Behzad (University of Illinois at Urbana-Champaign)
>== Affiliations ==
> * H. Joe Lee (The HDF Group)
> * Mike Folk  (The HDF Group)
> * Paul Ramirez (NASA JPL)
> * Chris Mattmann (NASA JPL)
> * Lewis John McGibbney (Stanford University)
> * Denis Nadeau (NASA NCCS)
> * Pedro Vicente (University of California at Irvine)
> * Babak Behzad (University of Illinois at Urbana-Champaign)
>=== Champion ===
> * Paul Ramirez <paul DOT m DOT ramirez AT jpl DOT nasa DOT gov>
>=== Nominated Mentors ===
> * Chris Mattmann
> * Paul Ramirez
>=== Sponsoring Entity ===
> * Apache Incubator
>Paul Ramirez

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message