FGDCMETA.AML

FGDCMETA.AML is an extremely simplified variant of DOCUMENT.AML that generates metadata files in ASCII text from Arc/Info geo_datasets in a UNIX or NT environment. File structure and content conform with the FGDC Content Standard for Digital Geospatial Metadata. The output template is intended to be customized to the individual user's needs (which requires rudimentary knowledge of AML). Resulting files are intended for use with other metadata tools that conform to the FGDC metadata standard (such as xtme, tkme and mp).

Table of Contents: FGDCMETA.AML v.1.2

What's New with version 1.2?
Files included with this distribution
What does FGDCMETA.AML do?
Installation and Setup
Why do we need yet another variant of Document.aml?
What is NOT included in the output from FGDCMETA.AML?
Links to fgdcmeta.aml Reviews
Need help?

Files included with this distribution of fgdcmeta.12.tar :

README - a text version of this html file
fgdcmeta.aml - Arc/Info AML program that generates metadata
fgdcmeta.12.tar - Tar file containing fgdcmeta.aml and README

What's New with version 1.2?

Virtually all the improvements included in version 1.2 were programmed by Curtis V. Price, U.S. Geological Survey, Rapid City, SD.

Windows NT is supported. Default metadata editors are Xtme for UNIX and notepad.exe for NT. Note: a Windows95/NT version of the xtme metadata editor (tkme) is now available from the USGS.
Default metadata editor settings can be overridden by an entry in the user's .arc file.
Every element in the default output template includes parenthetical text prompting the user as to the nature of the information that should be provided for that element.
Abscissa and ordinate resolution values are extracted from the fuzzy tolerance value (vector) or cell size (raster) if it is defined.
If no datum is defined in the data set, NAD27 is assumed and a message to this effect is written to the standard output (screen).
The Identification_Information -> Cross_Reference section has been added.
Online_Linkage elements have been added.
The format (indentation) of the Spatial_Data_Organization_Information section in the case of raster data has been corrected.

What does FGDCMETA.AML do?

Purpose

FGDCMETA.AML generates metadata files in ASCII text; file structure and content conforms to FGDC metadata standards. The output template is designed to be customized to the individual user's needs. Resulting files are intended for use with other FGDC metadata tools such as xtme, tkme and mp.

FGDCMETA <geo_dataset> { CREATE { TEMPLATE | MINIMUM } | EDIT }

FGDCMETA USAGE (for verbose usage notes)

The CREATE option: (this is the default)
Creates a new metadata text file named <geo_dataset>.met in the current workspace, but it will not overwrite an existing <geo_dataset>.met. The indicated <geo_dataset> can be located in a different workspace, but the output metadata file will still be written to the current workspace. The new text file is in a generic format which can be customized by editing the template section of the AML. If you edit the template, you should use mp (the Metadata Parser from FGDC) to insure that the format remains compliant with the FGDC metadata standard, formally known as the Content Standards for Digital Geospatial Metadata (CSDGM).

TEMPLATE is the default switch for the CREATE option. It creates a field in the metadata text file for several metadata elements common to many GIS data sets. Metadata fields contain either a value extracted from Arc/Info or a generic statement in parentheses that prompts the user for the nature of the information to be entered. Generic statements, for example Use Constraints and Distribution Liability, can be customized as necessary by editing the template section of the AML.

The MINIMUM switch for the CREATE option generates an output file that requires minimal additional information to be added by the user. It includes only the Identification and Metadata sections (sections 1 and 7) of the CSDGM, plus any other information that can be automatically generated. This option should only be used by those who are very familiar with the CSDGM and wish to build a custom metadata file by hand. Raw output alone from this option is not sufficient as a complete metadata file.

The EDIT option:
This option opens an existing metadata file in the default text editor. It checks for metadata files in the following order:

<geo_dataset>.met file in the current workspace
<geo_dataset>.nar in the info directory of the <geo_dataset>

Only the <geo_dataset>.met file is displayed. If <geo_dataset>.met does not exist in the current workspace, then the program checks for the existence of a <geo_dataset>.nar file that may have been previously created with DOCUMENT.AML. A message will indicate the presence of a narrative file from DOCUMENT.AML, but DOCUMENT.AML must be invoked to view the file.

Discussion

FGDCMETA.AML is a hacked variant of DOCUMENT.AML that gets rid of all the bells and whistles and simply creates or displays an FGDC compliant metadata text file. It is primarily intended to be used as the first step in generating new FGDC metadata. When used to create new metadata it will:

automatically gather Arc/Info and system specific metadata information (such as projection specifics, spatial domain, feature types and counts, user name, etc.),
write this information out to the appropriate fields in the metadata file,
generate a few other headings for the user to fill in with the appropriate information, and
open the new metadata file in a textedit window.

The editor used is set in the AML code. You can override this by setting the AML global variable .fgdcmeta$editr during your arc session or in a file named ".arc" in your home directory, for example:

&s .fgdcmeta$editr <editor_of_choice>

These files are intended to be used with the FGDC metadata tools xtme, tkme and mp, which rely heavily on proper indentation. Proper indentation allows xtme and mp to parse metadata files and perform automated checks of format and metadata content. This in turn allows the metadata to be indexed and searched, which is one of its primary purposes. When creating new metadata, please refer to existing metadata files for examples. Many examples are available from the FGDC.

Notes

It is better if a <geo_dataset> has a projection defined before running FGDCMETA. FGDCMETA.AML has powerful projection extraction and spatial domain (areal extent) calculation functions (that were retained from DOCUMENT.AML) that are rendered useless if no projection is defined.

The NAD27 datum will be assumed if the horizontal datum is not specifically defined in the Arc/Info data set.

Installation and Setup

FGDCMETA.AML will run as is (we hope), so there is no actual installation required. However, you will probably want to customize the AML code for your organization. There are two things to customize:

the metadata editor that will be used by FGDCMETA.AML, and
the output template.

First make a back-up copy of the AML, then open fgdcmeta.aml in the text editor of your choice.

Setting the text editor that FGDCMETA.AML will use:
Go to the SETVARIABLES routine of FGDCMETA.AML. This routine starts on or about line 289. The variable editr sets the system command that launches a text editor for the metadata file you want to CREATE or EDIT. Set this variable to the full system command needed to launch the text editor of choice. You may want to use a UNIX system text editor such as textedit, emacs or vi, or a PC editor such as notepad.exe, or you may want to directly specify xtme.

For example, to launch textedit on a UNIX system, the command would be similar to the following:

&sv editr = /usr/openwin/bin/textedit
Or, to launch vi on a UNIX system, set the variable like this. Note the extra command line parameters.
&sv editr = xterm -e vi
Or, to launch xtme instead, the command may look like this:
&sv editr = xtme
This default example sets xtme for UNIX and notepad.exe for Windows NT:
&if [extract 1 [show &os]] cn 'NT' &then &sv editr notepad.exe
&else &sv editr xtme
The exact string will depend on your system setup.
Once you have set the editr variable, save the file and test it (by creating a new metadata file) to make sure there are no problems when the AML opens the text editor.
Customizing the output template:
This is a little more involved. It's not terribly difficult, but it requires that you have a good grasp of AML script writing and that you have a good understanding of how the FGDC Content Standard for Digital Geospatial Metadata (CSDGM) production rules work.
You are urged to use the FGDC metadata tools mp (Metadata Parser) and xtme (Xt Metadata Editor) or tkme (NT users) to assist you in this operation. These tools will help you build a properly ordered, formatted and indented template.
Go to the TEMPLATE routine of FGDCMETA.AML. This routine starts on or about line 899. Once you have determined how you want you output from FGDCMETA.AML to look, simply remove from the existing template the text you don't want, and replace it with the text you do want. Here are some hints to keep in mind:
1. Whatever edits you make, you will find the world to be a better place if none of the lines in the output metadata file exceed 80 characters in length.
2. There are two styles of entries here; those that write a line of text where no variables are involved, and those that write a line of text that include a variable.
  The general form of a line without a variable is:
  &s ok [write %txtunit% [format %iN% 'Some sort of text goes here.']]
  The general form of a line that includes a variable is:
  &s ok [write %txtunit% [format %iN% [quote A_Heading: [locase %.some$variable%]]]]
  (The N in "%iN%" is the indentation level as set in the block of indentation variables starting on or about line 945.)
  It is suggested that you note the differences and conform to these conventions for two reasons:
  - uniformity, and
  - it works. As you know, quotes and unquotes in AML can be very pesky. Also, it is suggested that you don't mess with the lines that include a variable, as these provide the automatically generated information that (presumably) you want.
3. Watch that indentation! Proper indentation is essential if you intend to use the output metadata files with xtme, tkme or mp.
4. After you have completed your customization, check, double-check and recheck again to insure that the results go through mp properly before distributing this AML to your other users.

Why do we need yet another variant of Document.aml?

Document.aml is a metadata generation tool for Arc/Info that was created by staff of the Water Resources Division of the USGS and subsequently included with recent releases of Arc/Info. For the purposes of this section, it is assumed that the reader has some familiarity with the Arc/Info document atool.

Over the last three years, the Illinois State Geological Survey has used Document.aml as its primary metadata collection tool. In this time there have been five or six generations of the tool: USGS ver. 2.0.2; Arc/Info vers. 7.0.2, 7.0.3, and 7.0.4; Blmdoc from the BLM; and the most recent USGS release. (ESRI version 7.1.1 has subsequently been released but is not included in this review.) The concept of Document.aml is excellent and the program has proven useful for creating on-line documentation for individual Arc/Info data sets. However, the mechanics of the program have shown several problems, especially in generating FGDC formatted metadata. These problems can be attributed to three primary causes:

Document.aml was written prior to the advent of the FGDC metadata standard, and the two formats are dissimilar,
There have been several different versions of Document.aml in a short time, and not all are compatible with each other, and
Document.aml forces Arc to become a data entry interface, something it is not optimized to do.

Some of the specific problems encountered are:

Files made with an earlier version seldom translate exactly to a later version. With every new version of Arc/Info, re-editing of the narrative file and information in various fields of the input menus is invariably necessary.
Some versions have produced FGDC-type output without a problem, but others fail on this operation. Different versions fail due to different special AML characters in the narrative file.
Lines of text are frequently omitted from the narrative file. The reason for this is the 80-character limit for each line in the .NAR INFO file. Unfortunately, the 80-character limit is not enforced during the text input phase, and the result is inadvertent loss of text.
Sections of the narrative file are frequently omitted from the FGDC output (especially when files created by an earlier version are processed by a subsequent version.)
Document.aml consumes an unreasonably large amount of cpu time during the narrative file phase.
The Arc license is unavailable during narrative file phase because it is running the text editor.
The FGDC output includes multiple pages of LOG information that is redundant and not very informative, but which often accounts for the bulk of the metadata document.
The fact that the FGDC output must always be edited to include additional information not collected by Document.aml is the crux of the problem. It raises the question, " Why do all that data entry in Document.aml when it's so much more straightforward (and less cpu intensive) to do it in xtme or some other text-based editor?"

In addition, we understand that ESRI is developing a new metadata tool, which casts some doubt on the future development and support of Document.aml. Because of these problems, the ISGS has chosen not to continue using Document.aml for the collection of FGDC metadata. Document.aml does have, however, some impressive functions including:

a very useful metadata gathering engine that uses the AML &DESCRIBE function, and
a powerful block of code that determines projection information and calculates the spatial domain in decimal degrees.

Not wanting to forego the convenience of these functions, Document.aml was used as a template to develop a related AML called Fgdcmeta.aml. This tool retains the excellent automatic data gathering routines of Document.aml, but discards entirely any manual data entry within Arc/Info. All data entry is done in xtme or another editor. The new aml consists of line commands only; the several menus and forms have been removed. The main function is described in four steps:

The user issues the fgdcmeta command in Arc/Info.
Data that can be automatically gathered (DESCRIBE data, etc.) are collected.
The data are written to a user defined FGDC CSDGM template. This template can be created by the user using xtme or any other ASCII text editor.
The " skeleton file" (template with describe data) is ported out to xtme (or other text editor) for subsequent editing.

The approach used in Fgdcmeta.aml has the following advantages over Document.aml:

The Arc license is not constrained to text editing functions only. The user can continue to use the Arc session and write metadata simultaneously.
The time-consuming process of editing inside Arc/Info and then re-editing outside Arc/Info is avoided.
Understanding the relationship between Document.aml data entry fields and the associated FGDC CSDGM element fields is no longer required. Users need know only one metadata format.
Fgdcmeta.aml supports the continued use and development of the FGDC tools mp and xtme.
The output template is user defined, and so supports revisions to the CSDGM. It is flexible enough to evolve with FGDC metadata format as long as the CSDGM maintains its present structure. Re-ordering of elements and sections, for example, would be less problematic for this tool than for mp and xtme, because it uses those two programs for template definition.
It helps avoid the problem of the unknown evolution of ESRI metadata collection tools and support for Document.aml. Because this tool is extremely simple in concept, it will be useful as long as Arc/Info version 7 is supported (and possibly beyond, depending on how versions 8 and higher operate).

There are disadvantages to using Fgdcmeta.aml. It is not as robust as Document.aml. In its current form, it only supports a one-time query of DESCRIBE data from coverages, grids, and tins for the purposes of generating FGDC compliant metadata outside of Arc/Info. It does not write the metadata to an INFO file or directly support update of an existing metadata file.

Some specifics about what is not included in the output from FGDCMETA.AML

Some sections of the FGDC metadata standard that were included in DOCUMENT.AML are not included in FGDCMETA.AML.

There is no Arc/Info LOG information included in the output of FGDCMETA.AML. DOCUMENT.AML included this information as a series of Process Steps under the Data Quality Information section. It is not included in FGDCMETA.AML because it often takes up pages and pages of output, and, in the humble opinion of your author, is not very informative. Also, the LOG information is not necessary unless one has the Arc/Info data set, and if one has the Arc/Info data set, then one has the LOG.
There is no function supporting section 5.1 Detailed Description under the Entity and Attribute Information section. The CSDGM production rules state that it is acceptable to provide entity and attribute information as a Detailed Description (5.1) or as an Overview Description (5.2) or both. I have found it much more straightforward to document the meanings and values of entities and attributes in tabular form (i.e. Overview) than in the form required by section 5.1. DOCUMENT.AML, on the other hand, does an good job of meeting the requirements of section 5.1 if that is what you need.

Need help?

Feel free to contact me by fax or email.

Daniel O. Nelson
Associate Staff Geologist
Illinois State Geological Survey
615 East Peabody Drive
Champaign, Illinois 61821
USA
Fax: 217-333-2830
Email: nelson@isgs.uiuc.edu