Introduction; Report Organization
1999-08
Introduction
This report outlines the efforts of the SHPO offices of New Mexico and Wyoming to implement a common cultural resource database design. This collaboration is developing a common spatial data structure and implemented a prototype Geographic Information System based on the ESRI Spatial Database Engine and other supporting ESRI Technologies.
This report documents NCPTT-funded efforts aimed at the advancing State Historic Preservation Office (SHPO) Geographic Information Systems (GIS) in the western United States. Many largescale cultural resource inventories are operated by agencies other than SHPOs for example, Tribal HPOs, historical societies, universities, and museums, so our focus has been more on cultural resource information systems (CRIS) and particularly on GIS components of these systems — rather than on SHPO-specific data needs.
This project was prompted by the common experiences of several SHPOs in implementing GIS technology in large, transaction-heavy CRIS environments, combined with their desire to pool resources. Because many GIS implementation problems stem from the magnitude of both the archeological record and the demand for information common to these two states, project findings will be most relevant to other similarly situated Western states. Our solutions are undoubtedly scaleable to less “intensive” CRISs, but we suspect the Cost effectiveness of benefits may be
diminished.
Although the applicants for this grant were the New Mexico and Wyoming SHPOs, many other entities were involved. Most of the initial data-modeling tasks were conducted as part of a U.S. Geological Survey, Federal Geographic Data Committee (FGDC), grant to develop metadata and data content standards specific to cultural resources in the western United States (Appendix 1). Representatives from most western states and federal land managing agencies participated in one or both of the FGDC-sponsored workshops. Their contribution to this project has been significant. In addition, the University of New Mexico, Earth Data Analysis Center (EDAC), conducted comprehensive training on the FGDC metadata standards and documentation tools during the first meeting.
Project personnel consisted of the following individuals:
Report Organization
This is a technology implementation project. Presentation of all of the many technical details relating to our effort is exceedingly difficult in a report format. We have decided to present much of the technical information as appendices to this report, but many more details exist in application
code, database dictionaries, manuals, and the like. The main products of this grant are computer applications These too are difficult to appreciate from a report. All project materials and applications are, of course, available for review and demonstration, and we stand ready to assist
any agency that may wish to benefit from our experience.
The report first describes the problems we set out to solve in this project, our major objectives, and our strategy for reaching these objectives. Second, we provide a task-by-task summary of our efforts to date. Finally, we attempt to assess the cost effectiveness of what we have done and to
make recommendations on technology transfer.
GIS and SHPO Information Systems: “The Awful Truth”
Location is central to the management of cultural resources. If the location of a building, district, site, or object is unknown, then no action can be taken to manage, to preserve, to reconstruct, or to protect it. In spite of the central importance of location, spatial information technologies such as GIS are rarely integrated into everyday SHPO decision-making. As revealed in recent surveys of historic preservation archives in the US (Wood 1990, Ebert et al 1994, COAHP 1994), database management technologies are well established, but the transition to GIS technology has been very
slow in coming in spite of a very high user demand for geospatial data on cultural resources. Why?
Three reasons for slow adoption of GIS based on Wyoming and New Mexico SHPO’s experiences
are:
- GIS is not optimized for transaction-based computing environments typical among Western SHPOs. GIS is optimized for data analysis not data management. Current GIS technology provides efficient and robust tools for manipulating and storing spatial data, but the tools provided for data management are designed mainly to build and manage large, static, analytical datasets. Most SHPO information systems are transaction-based. That is, these systems are built and continuously updated through database transactions, consisting of property and investigation records generated by management and research activities. In western states, the number of annual database transactions generated by these activities ranges in the
tens of thousands. Compared to a modern RDBMS, GIS transaction processing controls–those functions that maintain the security and integrity of a multi-user database during simultaneous data entry, validation, and query events — are primitive. - Spatial relationships are difficult to create and maintain at the statewide level. Most GIS data models are organized in terms of space rather than the mapped features. This requires that feature topology be established at the time of data capture and then pre-stored in a proprietary file format (a coverage). Topology refers to the spatial relationships among connecting, adjacent, or overlapping elements (i.e., nodes, arcs, polygons, etc.) used, in this case, to rep resent cultural features (i.e., sites, buildings, districts, trails, etc) GIS is designed to efficiently store, manage, and manipulate these spatial relationships but, in GIS products like Arclnfo, the topological model necessitates special processing and storage in a proprietary file format (a coverage). Even more problematic, the topological model requires large databases to be partitioned into smaller spatial units (tiles) for processing and storage efficiency. Coverage tiles must be rebuilt every time new features are added or existing features are edited. Features crossing partition boundaries must also be edge-matched prior to storage. The creation and maintenance of topology requires considerable skill and training and cannot be performed effectively by untrained staff. Topology is responsible for the power and efficiency of GIS in manipulating spatial data, but it also introduces complexity and processing overhead to the basic data collection and transaction-processing functions that dominate SHPO information systems.
- Broad distribution of spatial data through GIS is difficult and expensive. Access to applications has been a major obstacle to widespread use of GIS technology. Paper maps are commonly used to distribute the results of GIS-based analysis widely, but real-time use of geographic data has been precluded by the requirement to have access to expensive and complex hardware and software. Even when GIS or desktop mapping applications are obtained, the use of spatial data beyond the local work group usually requires replication of the database or access to very high speed wide area network connections. If data replication is performed then highly structured procedures must be developed and followed religiously in order to safely manage multiple database copies.
During NMSHPO’s pilot GIS project, the ESRI Arclnfo environment was found to be very cumbersome. Data capture proceeded in batch mode, one map sheet at a time, while new in formation was placed in a backlog until the appropriate map sheet was processed. After multiple layers for each map sheet were digitized or scanned, extreme care had to be taken to associate database records with the appropriate manuscripted features, requiring considerable collaboration between the archeologist who created the manuscript and the GIS specialist. The entire data collection process was so procedurally complex and labor-intensive that we began questioning the cost-effectiveness of GIS. We found that GIS tended to intensify, rather than resolve, problems related to the three fundamental information system objectives: data capture, data management and data delivery.
We proposed a technical solution to NCPTT based on a promising new GIS technology: The Environmental Systems Research Institute (ESRI) Spatial Database Engine (SDE). The remainder of this report describes our efforts to implement and evaluate this technology. The objectives of the project are twofold:
- Develop a common logical spatial model for cultural resources among New Mexico, Wyoming, and other interested Western states.
- Develop a spatial database prototype, using SDE, in New Mexico based on that model.
In out proposal, several tasks and deliverable products were defined under each of these objectives. These are discussed below.
Chapter 1: Introduction, Project Goal, and Previous Studies
Objective #1: Develop a Common Logical Data Model for Cultural Resources
Three associated tasks addressed this objective:
- Model entities, spatial/non-spatial database links, and specify metadata
- Review model with partners and distribute model to outside reviewers
- Revise model based on comments
As mentioned in the introduction, all logical data modeling tasks were accomplished through a US Geological Survey grant to develop metadata and data content standards for cultural resources. This provided an opportunity to expand the collaboration to involve many more western states (and even some east of the Mississippi!) and generated considerable interest and support from federal land management agencies. Although the process of creating a formal data standard will involve several additional levels of review and take several years, we were able to create a solid foundation for current cultural resource GIS efforts at the New Mexico and Wyoming SHPOs. The preliminary report on the first FGDC workshop appears in Appendix 2 and is available on-line at:
Briefly, the FGDC workshop had three components. First, participants were introduced to the National Spatial Database Infrastructure concept and metadata documentation standards and tools. We were aided in this effort by the University of New Mexico, Earth Data Analysis Center. Second, the group focused on identifying basic cultural resource data entities and specifying key attribute (i.e., non-spatial) data for management. Finally and this was key to the success of this grant workshop participants developed a spatial data model for the major cultural resource data entities and identified key metadata items Owing to a widespread need to accommodate large amounts of highly variable legacy data in existing CRISs, this task might best be seen as a “best practices” guide rather than a data standard.
The following discussion briefly describes cultural resource data entities and their interrelationships as defined by the FGDC Workshop Best practices recommendations for spatial data representation and metadata are then presented.
Entity Definitions and Relationships
To minimize confusion, we have adopted National Register of Historic Places (NRHP) terms and definitions for historic property types:
“The National Register of Historic Places includes significant properties, classified as buildings, Sites, districts, Structures, or objects (NRHP Bulletin 15: p. 4).
Definitions for these five categories of historic properties are fully described in National Register Bulletin #15 and will not be repeated here Subsequent NRHP Bulletins have accommodated Historic and Cultural Landscapes (Bulletins 18, 30) and Traditional Cultural Properties (Bulletin 38), but these property types still fall within the original definitions provided in Bulletin 15.
To build a logical model it was necessary to focus on how historic property types are related to each other. The NRHP is not concerned with such relationships at a logical level. For example, buildings, sites, districts, structures, and objects are all considered as historic properties but districts had to be separated out from the other four historic property types to recognize and preserve the complex relationships that exist between districts and their constituent properties. to Only one additional major entity (Investigations) had to be added to the model to create a logical data model for cultural resource management.
Figure 1 illustrates a first order data model for cultural resources management. The model is built around three major data entities with geospatial references:
- Resource an individual building, structure, object, or site. A historic property constituting the smallest unit of management considered by the NRHP.
- Resource Aggregation a defined historic property consisting of a collection of two or more Resources related by proximity and/or a common theme An area, referred to as a district or landscape by NRHP, created to manage Resources contained within an explicitly defined area, or a set of dispersed but thematically related Resources; Resource aggregations may also be related to each other in a parent-child fashion, for example to link together historic districts associated with a common theme.
- Investigation: an event or activity resulting in the identification, documentation, restoration, rehabilitation or preservation of historic properties Investigations may, or may not (in the case of “negative” identification efforts), relate to one or more historic properties Common examples of investigations include inventory, excavation, documentation, and restoration activities.
Several minor entities relating to Investigations were also defined during the workshop (see Figure 2):
- Visit: the observational record relating a specific Investigation with a specific Resource or Resource Aggregation. When linked to a Visit, date-stamped observations on resource condition, status, and boundary definitions allow long-term maintenance of property “histories.” Visits relate properties to investigations in a many-to-many fashion, a property may be the focus of more than one investigation, and a single investigation may involve multiple historic properties. Visits insure that the integrity of these relationships are maintained.
- Investigation Aggregation: a collection of two or more Investigations related through a common, usually management-related, undertaking. This entity provides a reliable means of relating multiple investigation events or phases (e g., overview, inventory, data recovery, etc. ) with a larger undertaking (e g, a federal project or permit, a long-term research project) Undertakings may also be linked to other undertakings through a parent-child relationship.
- Publication: a report or other document describing a single investigation. This was determined to be a one-to-many relationship an investigation may produce multiple publications (or none), but a publication may describe only one investigation.
These three entities are considered minor because geospatial references were considered either optional (Visits) or not relevant (Investigation Aggregations and Publications) by workshop participants. Other entities relating to management processes were suggested during the workshop (e.g., Management Areas) but are not described here.
Non-Spatial Attributes
Non-spatial attributes for all major entities were considered at some length during the initial FGDC Workshop in Glorieta, NM, and then refined at a second meeting in Denver in February 1999. Given the primary emphasis of this project on spatial data models, we will refer readers to the on-line FGDC reports, rather than reproduce this information here.
Spatial Representation
As stated earlier, the need to accommodate legacy data necessitated a “best practices” approach. The problems of legacy data are perhaps most critical when spatial data are considered. Many important historic properties have been located without a great concern for source scale or positional accuracy. Maps have gotten better over the years and new technologies, such as the Global Positioning System, make spatial representation easier and more accurate. Management needs dictate that less accurate old data be utilized until updated locations can be obtained, so the accuracy and reliability of this data must be documented through metadata Our efforts were aimed at meeting these needs.
Best practices dictate that cultural resource entities be represented as follows:
- Minimal: centroids or line segments. This option is most appropriate for legacy data where information on size and/or shape is either unknown or unreliable. Also appropriate for very small cultural resources that cannot be represented accurately at the scale of the source graphics (e.g., largest resource dimension is less than National Map Accuracy Standards).
- Better: buffered points or lines. Resource boundaries are “calculated” by buffering a centroid or line segment with some estimate of resource size (e g., area, length, width).
- Even Better: minimum bounding rectangle. Resource boundaries are roughly approximated by a rectangle.
- Best: boundary polygon. Resource boundaries accurately represented by a polygon.
Best practices also indicate the need for a great deal of flexibility in how cultural resources are represented. To wit:
- cultural resources may overlap spatially.
- a single cultural resource entity may have multiple boundaries definitions relating to separate investigation events (e.g. , redefinitions of archeological site boundaries).
- a single cultural resource entity may be represented as the union of multiple objects and object types (i.e., points, lines, or polygons; e.g., an archeological inventory of an oil well pad and associated access road, a historic trail and associated buildings).
- a single cultural resource entity may have different types of boundaries (e.g., National Register vs. State Register boundaries; legal vs. traditional boundaries.
The implications of these facts for the design of a GIS or database are significant. Cultural resource location and configuration can be a complex matter and feature representation must take many factors into consideration. The most important decisions are related to how the information will be used. What are the data needs of CRIS system users? A national database of National Register Properties can probably rely on simple point and line locations at a fairly gross scale, but a state or local CRIS may need accurate property boundaries and large scale base maps to’ be able to make many planning decisions (e.g., “is this trench going to affect the county courthouse?”). Whatever level of accuracy is appropriate, the need for comprehensive spatial metadata is critical. When legacy data are involved, data should be maintained at the level of the individual feature (e.g., “cultural resource X was located using GPS its location is accurate to within 10 meters”).
Recommended locational methods and associated metadata are as follows:
- Minimal: map-derived coordinates based on UTM or State Plane coordinates, Latitude/Longitude, etc. Metadata source map identification, scale, date; Coordinate system zone, datum. (Note the Public Land Survey System (PLSS) is not a locational system, but some institutions use “Township/Range/Section/Aliquot units” to locate cultural resources –this is not recommended, but it is better than nothing! The PLSS Meridian must be included if this system is used).
- Better: Global Positioning System (GPS)-derived coordinatesbased on UTM or State Plane coordinates, Latitude/Longitude, etc. Metadata: estimate of positional accuracy (e g, Standard Deviation = ± >100, 10-lOOm, 1-lOm,
- Best: Cadastral survey or parcel map coordinates based on UTM or State Plane coordinates, Latitude/Longitude, etc Metadata estimate of positional accuracy.
Objective #2: Develop a Spatial Database Prototype in New Mexico based on the Logical Data Model
Objective #2: Develop a Spatial Database Prototype in New Mexico based on the Logical Data Model
The NMCRIS spatial database prototype consists of a database server component, serving as the main data repository, which is connected over local and internet computer networks to multiple client components (applications) used for data capture and query (Figure 3). Our implementation strategy divided the effort into three overlapping sets of tasks focusing on the spatial database server and two application development tasks:
- Creating the spatial data server
- Developing the data capute application
- Developing the spatial query application
Table 2 describes these tasks in detail.
Creating the Spatial Data Server
During the early project phases, emphasis was placed on training staff and creating a stable test environment for the prototype separate from the production NMCRIS. Installation of SDE was trouble-free. Following the translation of our logical data model to a physical model in Oracle and SDE, existing spatial data was then converted from Arclnfo to SDE. As applications were developed for data capture, the ARMS staff was able to immediately use SDE for query and analysis tasks using ArcView as the interface to SDE.
Many decisions had to be made in creating the physical design and integrating SDE with NMCRIS. Some of the more important ones are listed below and are documented in Appendix 3:
- What SDE layers are required to represent cultural resources?
- How should SDE layers be related to existing database tables?
- What metadata, if any, should be stored along with each feature?
- What coordinate system and projection will be used?
- How will features be represented? What “shape masks” will be used in each layer?
- How should the SDE tables be populated?
- What is the appropriate spatial index grid size for each layer?
- How will average feature size (no. of vertices) affect database space allocations?
- Who should have access to SDE-related database tables and for what functions?
These decisions and many others were postponed until ARMS technical staff had completed the first round of training in Redlands CA (“SDE Administration”) and the logical data model was developed. Training was absolutely essential. Most decisions also benefit from a comprehensive understanding of the underlying database structure and the Oracle RDBMS NMCRIS has been operational for over five years so we were not faced with major database design issues we just had to create new SDE layers and relate them to our existing Oracle tables. Having a well thought out logical data model from the first FGDC workshop was also essential to a smooth implementation.
SDE modifications to the NMCRIS physical database design are illustrated in Figure 4. Table 3 lists the five active layers created to represent Resources (archeological sites) and Investigations (archeological surveys) A layer was also created to represent USGS Quadrangle boundaries used mainly for query purposes. In the future ARMS will provide additional layers for non-archeological Resources, Resource Aggregations and other entities. Appendix 3 provides details of the content and structure of the five main SDE layers.
Developing Applications
Application development efforts first focused on the ArcView data capture tool. This application, allows the ARMS staff to perform on-screen (“heads-up”) digitizing on a report-by-report basis. The primary design goal for this application was to provide a means for non-technical staff to quickly capture geospatial data with minimal training and disruption of work flow. The application thus closely follows current processing procedures where reports are processing in serial fashion and spatial data are transferred from the source graphics in each report to a master Set of USGS 7.5’ topographic maps. Appendix 4 presents a description and illustration of this tool.
During testing of this application, ARMS staff realized significant gains in productivity. We estimate this gain to be 2 -10 times greater productivity over our established digitizing methods, depending on the complexity and density of features involved. These gains are so significant that the test system has since become our production system for survey area data collection.
After basic data capture functionality was established, ARMS technical staff prepared system requirements and a scope of work for ESRI–Boulder to develop a text-based query application using SDE. This application uses the SDE “C Application Programming Interface” to spatially enable a simple text-based query tool that has been in use for the past 8 years by ARMS staff and outside users. The basic query program runs on one of the ARMS UNIX servers and is accessed via Telnet and dial-up connections. This extension greatly expands the utility of this simple text- based tool by using SDE technology to resolve fairly complex spatial queries and generate reports. Previously, users were able to enter a single rectangular query area defined by four UTM coordinates to query archeological site locations and could only retrieve information on previous surveys through USGS 7.5’ quadrangle units. Using SDE, users will now be able to specify any number of query areas in any configuration, define a buffer zone, and automatically receive information about archeological sites, surveys, and other investigations taking place within their query area. This insures that comprehensive pre-field records checks are conducted. In addition, SDE automatically reprojects UTM coordinates between New Mexico’s two Zones an operation that also previously required two separate queries. Appendix 5 provides some insight into the functionality of this component. The query logic built into this application will be an essential part of many other business processes at ARMS and we expect it will be integrated into many future applications.
Choices, Choices, Choices
SDE provides a robust environment in which to capture manage and distribute complex spatial data It does this mainly through the underlying RDBMS: spatial data, indexes, and other related data are stored and maintained directly in RDBMS tables. But SDE is of little utility without client applications to enter, edit, and distribute information.
There are many technical directions and these are changing almost on a daily basis. Table 4 lists some of these choices along with some pros and cons of each. We have made our decisions in this area based on our development schedule, our available technical skills, and our existing information system resources. We picked ArcView/Avenue for data capture mainly because it can access SDE right out of the box and the digitizing functionality is built in. We were thus able to concentrate on the actual SDE transactions rather than worrying with user interface issues ArcView requires a significant amount of desktop computing power and some training, but for us these costs do not outweigh the time required to develop a new user interface. We would still need ArcView for query and analysis purposes, so ArcView was the logical choice.
In the long run, we will probably use several different tools for data distribution. We are relying on the “C” API for our basic query tool in order to leverage SDE’ s query capabilities as widely and quickly as possible and with minimal end-user training costs. We were able to use consultants to develop the spatial query program, thus allowing us to concentrate on the data capture applications in-house. In the future, we will be developing graphic query tools so that we can archive our fragile paper maps and make spatial data more accessible to cultural resource managers over the internet.
Evaluation
In our proposal, we stated that “GIS has the potential to fundamentally change many SHPO operations” but that to be successful, the technology “must be fully integrated into the daily work routine of SHPOs… before these kinds of fundamental changes can be realized.” GIS must “help solve basic information systems functions relating to data collection, quality assurance, and data delivery, or the added expense of GIS may be difficult to justify.”
It is expensive. Historically, it has been hard to justify. We believe SDE justifies the expense. Our experience gained in the course of this project confirms our original suspicions: SDE solves problems in the capture, management, and distribution of cultural resource information. Spatial data collection has been one of the biggest obstacles for NMCRIS. – The process is now greatly simplified over our previous approach. SDE allows an efficient transactional approach to data capture, uncomplicated by feature topology or the need to subdivide space into manageable units. With SDE, sites, districts, and buildings — rather than space — are the central organizing principle. This allows us to maintain a more logical and efficient work flow, and requires far less training, than our previous approach to spatial data capture. Put simply, we can be more productive with SDE.
Spatial data management functions are handled by the underlying RDBMS — a very mature and robust technology. All records — spatial and non-spatial — are inserted, modified, indexed, and deleted in the same database environment allowing, for example, automatic recovery of digitized site boundaries following a system crash. Administrative costs for spatial data management are thus rolled into our overall RDBMS administration, resulting in significant savings. Moreover, the integrity and security of spatial data are greatly increased in the RDBMS environment.
SDE-based data distribution options are many and diverse. Powerful client applications are available out-of-the-box with ArcView or may be developed in a wide variety of environments. Effective SDE- based solutions can range from simple text reports, as in the NMCRIS SDE enhanced query program, to custom map server applications requiring that the user have only a Web Browser and internet connection. Combined with SDE, these tools are the key to integrating spatial data into the daily work routine of cultural resource managers and finally realizing true value from GIS technology.
We have yet to see any serious technical obstacles to using this technology. Could it be improved? You bet! There are several areas in need of improvement. First of all, it is too expensive. Competing products and increasing economies of scale are needed to bring this technology down to an affordable level. This is already happening. SDE’ s reprojection and administrative tools could also use some enhancements and we’d like to be able to use SDE to handle raster data such as USGS Digital Raster Graphics, as well as vector data. The good news is that these, and many other improvements, will probably be implemented in the next version of SDE and possibly in competing products.
Cost may be the most significant liability of the technology. SDE and RDBMS still require a fairly substantial investment. Initial costs for hardware and software are much lower than when ARMS invested in RDBMS almost a decade ago, but the figures are still substantial for a small state agency. Annual maintenance costs are substantial, but recruiting and retaining skilled technical staff is an even bigger expense. Adult supervision is required SDE and RDBMS technology should not be implemented without skilled technical staff or consultants. Training is essential to retain staff and keep up with technology nobody or at least nobody you can afford has all the required technical skills. Consultants and partnerships with other better technically endowed agencies should also be considered.
There are also some indirect benefits to consider. Because SDE allows location to be fully integrated into NMCRIS, we have begun to see more errors. Many of these are simple locational errors (“these coordinates are in the wrong UTM zone!”) and are easy to fix once isolated, but having site and survey locations at ones fingertips has revealed more logical errors (e.g., “what is this Anasazi site doing in Catron County?”) The long-term result: better error trapping and more reliable data.
Many other phenomena have spatial dimensions besides sites and surveys. Ecological zones, soil types, and land ownership, for example, are often cited as critical in an archeological CRIS SDE provides a relatively straightforward method for utilizing these data for query and analysis without requiring observations in the field and another data item on the site form. Accurate, consistent spatially referenced data are widely available and are getting better every day. The result better analyses and better management decisions.
In summary, we have found through direct experience that SDE solves most difficult problems surrounding spatial data in a CRIS environment. The investment is substantial. For New Mexico and other “high-volume” states the expense of implmenting GIS using SDE is proportional to its benefits.
References Cited
References Cited
COAHP (Colorado Office of Archeology and Historic Preservation)
-
1994 Cultural Resource Programs Compilation of programs of Rocky Mountain and Great Plains States Colorado Historical Society, Office of Archeology and Historic Preservation
Ebert and Associates, Inc.
-
1994 Generally Applicable Methods and Techniques for Conversion of Existing State Cultural Resource Archives to Geographic Information Systems Databases Proposal submitted to National Science Foundation
National Park Service
-
nd How to Apply National Register Criteria for Evaluation, National Register Bulletin 15
-
nd How to Evaluate and Nominate Designed Historic Landscapes, National Register Bulletin 18.
-
nd Guidelines for Evaluating and Documenting Rural Historic Landscapes, National Register Bulletin 30.
-
nd Guidelines for Evaluating and Documenting Traditional Cultural Properties, National Register Bulletin 38.
Wood, Noriko
- 1990 Computer Use in State Historic Preservation Offices Interagency Resources Division, Natiopnal Park Service, Washington, DC.
Appendix 1: FGDC Press Release
The United States Geological Survey Federal Geographic Data Committee (FGDC) has awarded $32,150 to the Wyoming State Historic Preservation Office to develop a metadata standard for cultural resources in the western United States. Cultural resources consist of archeological and historical sites, buildings, and historic districts. The Earth Data Analysis Center (EDAC) and the New Mexico SHPO are principal participants and co-sponsors of this project.
Metadata are “data about data.” Metadata describes the content, quality, condition, and characteristics of digital data sets. Metadata are used by potential data users, often through spatial data clearinghouses accessed over the Internet, to determine whether or not the information is appropriate for their application, and how to obtain the dataset. Metadata helps to protect an agency’s investment in digital data.
The project will be conducted in conjunction with historic preservation agencies in eight other western states’ the New Mexico State Historic Preservation Office, the Colorado State Historic Preservation Office, the University of Montana Department of Anthropology, the Montana State Historic Preservation Office, the Archaeology Division of the Arizona State Museum, University of Arizona, the Nevada State Museum, the Idaho State Historic Preservation Office, the Utah State Historic Preservation Office, and the California Office of Historic Preservation.
The project will establish a draft metadata standard to be reviewed by the FGDC Standards working group and will coordinate the development with the FGDC’s Cultural And Demographic Subcommittee. The project will build upon the current FGDC metadata standard and the supplement for geospatially referenced cultural and demographic data metadata.
A training session involving cultural resource managers and archivists will be conducted for the project participants, followed by work on pertinent data elements to include in a cultural resources metadata file. A draft standard document will be prepared for FGDC and public review.
The training session will be conducted by EDAC at the University of New Mexico. EDAC has extensive experience in facilitating and providing training in this area.
This effort is an open collaborative project sponsored by the western States. We encourage those who are interested in participating in the grant work to contact Mary Hopkins (WYSHPO) at 307-766-5324, hopkins@uwyo.edu or Tim Seaman (NMSHPO) 505-827-6347 ext. 531, seaman@arms.state.nm.us.
Appendix 2: FGDC Preliminary Report
http.//colby.uwyo.edu
Appendix 3: SDE Design Details Layer Documentation
ARCH_SURVEY_SMALL (small survey locations)
ARCH_SURVEY_LINE (linear survey locations)
ARCH_SURVEY_POLY(survey boundary polygons)
Attribute Fields:
- ACTIVITY_NUM (key)
- ARMS_ACTIVITY_NUMBER (external key)
Metadata:
- ARCH_SURVEY.USGS_75_TOPO_SGRAPHIC_FLG
- ARCH_SURVEY.GPS_SGRAPHIC_FLG
- ARCH_SURVEY.OTHER_TOPO_SGRAPHIC_DESC
- ARCH_SURVEY.RECT_AERIAL_SGRAPHIC_DESC
- ARCH_SURVEY.UNRECT_AERIAL_SGRAPHIC_DESC
- ARCH_SURVEY.SKETCH_MAP_SGRAPHIC_DESC
- ARCH_SURVEY.OTHER_SGRAPHIC_DESC
Shape Masks:
ARCH_SURVEY_SMALL: polygons only multiple (unioned) polygons OK
ARCH_SURVEY_LINE: lines only multiple (unioned) lines OK
ARCH_SURVEY_POLY polygons only– multiple (unioned) polygons OK
Index Grid Size: 25,000 meters
Coordinate System: UTM Zone 13: NAD27
ARCH_SITE_CENTROID (Site Centroids)
Attribute Fields:
- ARCH_SITE_CENTROID.ARCH_SITE_NUM (internal key)
- ARCH_SITE_CENTROID ARMS_ARCH_SITE_NUMBER (external key)
Metadata:
- ARCH_SITE_LOCATION.USGS_75_TOPO_SGRAPHIC_FLG
- ARCH_SITE_LOCATION GPS_SGRAPHIC_FLG
- ARCH_SITE_LOCATION.OTHER_TOPO_SGRAPHIC_DESC
- ARCH_SITE_LOCATIONLRECT_AERIAL_SGRAPHIC_DESC
- ARCH_SITE_LOCATION UNRECT_AERIAL_SGRAPHIC_DESC
- ARCH_SITE_LOCATION.OTHER_SGRAPHIC_DESC
Shape Mask: single points Only
Index Grid Size: 100,000 meters
Coordinate System: UTM Zone 13 NAD27
Required update functions and dependencies:
1-to-1 relationship with ARCH_SITE_LOCATION (mandatory relation for all rows with valid UTM coordinates)
with ARCH_SITE_LOCATION (Registration Application must add new record; changes in ARCH_SITE_CENTROID will require update of ARCH_SITE_LOCATION).
- ARCH_SITE_LOCATION.UTM_ZONE (reprojection required for Zone 12)
- ARCH_SITE_LOCATION. UTM_EASTING (reprojection required for Zone 12)
- ARCH_SITE_LOCATION.UTM_NORTHING
ARCH_SITE_BOUNDARY (Site Boundaries)
Attribute Fields:
- ARCH_SITE_BOUNDARY ARCH_SITE_NUM (internal key)
- ARCH_SITE_BOUNDARY ARMS_ARCH_SITE_NUMBER (external key)
- ARCH_SITE_BOUNDARY ACTIVITY_NUM (foreign key ACTIVITY; nullable for legacy data.]
Metadata:
- ARCH_SITE_BOUNDARY.BOUNDARY_TYPE (see illustration):
NULL– null shape (no length available boundaries undefined)
LEGACY buffered centroid (length=buffer radius)
SMALL buffered centroid (length - ARCH_SITE_BOUNDARY BOUNDARY_SOURCE:
7 5’_TOPO 1:24,000 scale USGS topographic map
GPS_POOR –positional accuracy (std dev.) >100 meters
GPS_GOOD –positional accuracy (std dev) 10- 100 meters
GPS_EXCELLENT positional accuracy (std dev.) 1 – 10 meters GPS_SUBMETER positional accuracy (std dev) < 1 meter
SKETCH_MAP specify scale in BOUNDARY_SOURCE_NOTES OTHER_SOURCE identify source and scale in BOUNDARY_SOURCE_NOTES - ARCH_SITE_BOUNDARY.BOUNDARY_SOURCE_NOTES (text field)
- ARCH_SITE_PHY_DESC.SITE_BOUNDARY_COMPLETE_FLG
- ARCH_SITE_PHY_DESC.SITE_BOUNDARY_INCOMPLETE_FL
- ARCH_SITE_PHY_DESC.INCOMPLETE_SITE_BOUNDARY_DESC
Shape Mask: polygons only multiple (unioned) polygons OK
Index Grid Size: Undetermined
Coordinate System: UTM Zone 13: NAD27
Required update functions and dependencies:
Changes in ARCH_SITE_BOUNDARY will require changes in ARCH_SITE_PHY_DESC when BOUNDARY_SOURCE= GPS_G00D (or better) or SKETCH_MAP:
- ARCH_SITE_PHY_DESC.MAXIMUM_SITE_LENGTH
- ARCH_SITE_PHY_DESC MAXIMUM_SITE_WIDTH
- ARC_SITE_PHY_DESC.SITE_AREA
- ARCH_SITE_PHY_DESC.SITE_DIMENSIONS_MEASURED_FLG (not null)
- ARCH_SITE_PHY_DESC SITE_AREA_MEASURED_FLG (not null)
Changes in ARCH_SITE_BOUNDARY will require recomputation of ARCH_SITE_CENTROID coordinates
NMCRIS Site Boundary Types
Appendix 4: SDE Design Details — ArcView Data Entry Application
Menus all retain original functions and command sets
Buttons retaining original function:
Save
Help
Zoom to active theme Zoom out
Zoom in
Clear selection
Buttons with customized functions:
Connect
- :Fires off the master AVENUE script and connects to SDE
Zoom to Active DRG
- : Zooms view to the extent of active DRG(s)
Edit
- : Clones the active SDE theme into a theme named “Checked Out Features”
Union
- : Unions all features based on a common external Id number
Verify
- : Invokes an interactive process the get user input if the external ID number then performs a SQL query to return a verification report
Save $
- : Commits the changes to the database
Undo Edit
- : Removes any changes made to the “Checked Out Features” theme and deletes the theme from the view.
Clear View
- : Clears the view of all themes
Erase
- : Combines the Undo Edit and the Clear View buttons
Tools retaining original functions:
Identify Select
Vertex edit
Zoom In
Zoom Out
Pan
Drawing tools
- Point
- Line
- Split line
- Polygon
- Split polygon
- Append polygon
Tools with new or modified functions:
Select by rectangle:Modified to keep the selection rectangle graphic showing in the view
Select by polygon:Modified to keep the selection polygon graphic showing in the view
Drawing tools Diamond:Creates a diamond shape for representing surveys under 2.5 acres.
SDE Editing Schema
1. User selects USGS quad(s) from pick list
2. If not previously copied locally, user is prompted to load appropriate CDROM and the quad is copied locally.
3. User logs in to SDE
4. User selects the SDE layer(s) to be edited
5. Application spatially selects features from the SDE layer(s) based on the merged extent polygon of the selected USGS quad(s)
6. User selects the SDE layer to checkout for editing
7. Application creates a spatial lock to prevent editing conflict, clones the recordset for editing, and loads the cloned theme into the view.
8. User zooms to the area of interest.
9. User performs the edit
- Add new feature
- Digitizes new feature with the appropriate drawing tool
- Attribute and verify the record
User enters the external ID number
Application returns a report of the record for verification
If verification is correct, application attributes the internal ID number - If necessary, user unions multiple shapes into one record
- Modify existing shape Same as adding shape
- Deleting shape
User selects and deletes exiting shape
10. User saves edits and application commits transaction to database
- Application checks and verifies record and if ok commits
- If not ok then record is written to error shapefile
11. User either ends session and disconnects from SDE or selects next SDE layer to edit by looping back to step 6
Appendix 5: SDE Design Details — “C” API Query Application
NMCRIS Spatial Database Access Program Testing Notes
ASSUMPTIONS:
- The owner of all layers is sde, the owner of all non-sde tables is ops$nmcris
- All users will have $SDEHOME/Iib added to their LD_LIBRARY_PATH environment variable
- All testers will have SELECT,UPDATE,INSERT,DELETE access to the following sde layers:
INPUT_POINTS, INPUT_LINES, INPUT_POLYGONS, INPUT_POINTS 12, INPUT_LINES 12, INPUT_POLYGONS 12, BUFFERED_POLYGONS, BUFFERED_POLYGONS 12, CLIPPED_POLYGONS, QUERY_POLYGONS, REPROJECTED_POLYGONS - All users and testers will have SELECT access to the following tables and layers:
SURVEY_SMALL, SURVEY_LINE, SURVEY_POLY, SITE_POINT, ARCH_SITE_LINK, QUAD_INDEX,
ARC_SURVEY_QUAD. - The users unix account name is the same as their database user name.
TESTING INSTRUCTIONS
- To create an executable, cd to the source code directory and type
- % make: which runs the Makefile.
Type
- % make clean: to erase all .0 files and the executable and rebuild the executable
- Alter Makefile for testing or operations
- To set up for testing mode: CFLAGS = -Dunix ${INCLUDE} -DTESTING
- for normal mode remove –DTESTING CFLAGS = -Dunix ${INCLUDE}
- Start the program by typing
- % spatial: for interactive data input
- % spatial: for data input from an input file
- If you are doing interactive input, the program will save user input into a file called inputData.txt. The
program will check if the inputData txt file already exists If the file exists, the user will be asked if they want
to overwrite the file. It will exit if they type anything but a word starting with ‘or a carriage return - The program check for the following:
Main Menu
- Valid Letter Typed (1-6)
- Letter Typed
- Zone
- Valid zone typed “12″ or “13″
