Data and File Structures

1
Data and File Structures

• CS 189
• Sept 11, 2007

2
Outline

• File Structures
• Vector Data Structures
• Raster Data Structures
• Single-Band
• Multi-Band
• Attribute Data Structures
• Data Management
• Map Documents
• .mxd file structure
• Absolute and Relative Paths
• Layers and Layer files (.lyr)

3
GIS File Structures

• Many, many, many different structures
• Shapefiles (ArcView 3.x, ArcGIS)
• Geodatabase (Feature Class) (ArcGIS)
• ArcSDE geodatabase (RDBMS Oracle)
• Personal geodatabase (.mdb)
• File geodatabase (.gdb)
• Coverages (Arc/Info Workstation)
• GRID (raster Arc/Info Workstation)
• Raster Catalog/Dataset (ArcGIS)

4
Vector File Structures

• Shapefile
• Developed by ESRI in the early 1990s (ArcView 2)
• A homogeneous collection of features that can
  have either point, multipoint, polyline, or
  polygon shapes (only one shape type per
  shapefile)
• Composed of three main files (mandatory) that
  contain spatial and attribute data
• .shp - stores the feature geometry
• .shx - stores the index of the feature geometry
• .dbf stores the attributes
• Optional files
• .prj - stores the coordinate system information
• .shp.xml stores metadata for the shapefile
• .avl ArcView Legend file
• .sbn and .sbx - spatial index of the features
• .fbn and .fbx - spatial index of features for
  read-only shapefiles
• .ain and .aih - attribute index of active fields
  in theme's attribute table
• .atx - attribute index for the .dbf file -
  ltshapefilegt.ltcolumnnamegt.atx

5
Vector File Structures

• Shapefile (Cont.)
• Demo
• Look at a Shapefile in with ArcCatalog
• View the same location in windows explorer (My
  Computer)
• Copy just the .shp file into a new location and
  try to view in ArcCatalog
• What happens? Why?
• Copy remaining files to new location
• Preview the table associated with the shapefile
  in ArcCatalog
• Open the .dbf file (from windows explorer) in MS
  Excel
• Compare the .dbf table in Excel to the table in
  ArcCatalog
• Whats different?

6
Vector File Structures

• Geodatabase
• The ESRI version was introduced with ArcGIS
  (ArcView/ArcInfo 8)
• Geodatabases are relational databases that
  contain geographic information.
• Geodatabases contain feature classes and tables.
• Feature classes store geographic features
  represented as points, lines, polygons.
• Feature classes can be organized into feature
  datasets.
• All feature classes in a feature dataset share
  the same coordinate system.
• Tables may contain additional attributes for a
  feature class or geographic information, such as
  addresses or x,y,z coordinates.
• Personal Geodatabases are stored in Microsoft
  Access.
• Multi-User Geodatabases may be stored in one of
  many commercial RDBMS including Oracle, MS
  SQLServer, IBM DB2 and Informix. ArcSDE is used
  to access and work with data in a commercial
  RDBMS from ArcGIS Desktop.
• Structure still being refined, significant
  changes may continue?

7
Vector File Structures

• Geodatabases (Cont.)
• Demo
• Look at a Personal Geodatabase in with ArcCatalog
• View the same location in windows explorer (My
  Computer)
• Open the Access database (.mdb) to see what
  tables it contains
• Preview the table associated with the Personal
  Geodatbase in ArcCatalog
• Attempt to demo a connection to a multi-user
  Geodatabase?
• Notice the difference in the structure of the
  different Geodatabase types?

8
Vector File Structures

• ArcInfo Coverages
• Hold-over from ArcInfo Workstation
• Limited support for viewing this format continues
  in ArcGIS
• Data structure comparable to that of the
  geodatabase, but more constrained.
• Coverages are made up of a set of files within a
  folder (the folder has the same name as the
  coverage)
• An INFO folder, which holds some catalogue
  information about the coverage, sits at the same
  level as the coverage folder.
• While this data format is being phased out, you
  do continue to see it quite a bit, and
  understanding the structure provides some insight
  into the ESRI GRID (raster) format.

9
Vector File Structures

• ArcInfo Coverages (Cont.)
• Demo
• Look at an ArcInfo Coverage with ArcCatalog
• View the same location in windows explorer (My
  Computer)
• Copy the coverage folder to a new location
  without copying the INFO folder.
• Try to view the new coverage in ArcCatalog.
• Copy the INFO folder over and try viewing it in
  ArcCatalog again.

10
Raster File Structures

• ArcInfo (ESRI) GRID
• Also a hold-over from ArcInfo Workstation.
• Support for viewing this format continues in
  ArcGIS.
• From the ESRI Knowledgebase
• Arc/INFO GRID format rasters are unique in
  their file structure. GRID rasters are stored in
  workspaces. A workspace contains one info
  subdirectory and a subdirectory for each GRID.
  Each GRID subdirectory contains several files
  that store geographic location and attribute data
  for the corresponding grid. The info subdirectory
  contains several files which maintain certain
  information about the GRID and other datasets,
  such as coverages, in the workspace.
• http//support.esri.com/index.cfm?faknowledgebase
  .techarticles.articleShowd30616
• GRID files contain single band raster data sets.
• GRID format is typically used to store elevation
  data.
• GRID files may be viewed in the core ArcGIS
  software, but for most GRID analysis tools, the
  Spatial Analyst extension is needed.

11
Raster File Structures

• ArcInfo (ESRI) GRID (Cont.)
• Demo
• Look at a GRID with ArcCatalog
• View the same location in windows explorer (My
  Computer)
• Copy the GRID folder to a new location without
  copying the INFO folder.
• Try to view the new GRID in ArcCatalog.
• Copy the INFO folder over and try viewing the
  GRID in ArcCatalog again.

12
Raster File Structures

• ERDAS Imagine (.img) Format
• Developed by ERDAS (now Leica Geosystems) for use
  with the ERDAS Imagine image processing software.
• Support for viewing this format continues in
  ArcGIS.
• From the ESRI Desktop Help
• Produced using the IMAGINE image processing
  software created by ERDAS. IMAGINE files can
  store both continuous and discrete single-band
  and multiband data.
• Imagine format is frequently used to store
  satellite imagery.

13
Raster File Structures

• ERDAS Imagine (.img) Format (Cont.)
• Demo
• Look at a .img file with ArcCatalog
• Open the .img in ArcMap
• Double click on the layer name in the Table of
  Contents (in ArcMap) to open the Layer Properties
  Dialog.
• Go to the Symbology tab in the Layer Properties
  Dialog.
• Change the band combinations so that R,G,B
  4,3,2.
• Copy and paste the layer file so that you have
  two copies in the Table of Contents.
• Change the band combinations of the new later so
  that R,G,B 1,2,3.
• Toggle the layer visibility to see the area
  represented differently.

14
Attribute File Structures

• Text Formats
• Comma Delimited (.csv)
• Tab Delimited (.txt)
• dBase format (.dbf)
• INFO files
• Access databases (.mdb)
• Excel Files (.xls)
• There are a number of ways to store the data on
  the computer, but these all represent attribute
  data as tables containing row (records) and
  columns (fields).

15
Attribute File Structures

• Create a basic comma delimited table in notepad.
• Dont forget to provide names for each field in
  the table. No spaces or odd characters in field
  names.
• Create another basic tab delimited table in
  notepad.
• After saving the files, open and examine them in
  ArcCatalog.
• Connect to a .xls file using ArcCatalog.

16
Non-GIS Exclusive File Structures

• CAD data files (.dxf, .dwg)
• GPS software (.ssf, .gps, .loc, etc.)
• Excel spreadsheets (.xls)
• dbase files (.dbf)
• Text file (tab, comma, etc. delimited) (.txt)
• Database table
• Microsoft Access
• RDBMS (Oracle, SQL Server, etc.)
• http//webhelp.esri.com/arcgisdesktop/9.2/index.cf
  m?TopicNameData_formats_supported_in_ArcGIS
• http//spatialnews.geocomm.com/education/av_fileex
  tensions.html

17
Data Management

• About Map Documents (.mxd files)
• In a typical work environment this is where you
  store your work at the end of the day. The .mxd
  file will store collections of data references
  and symbology created within ArcMap.
• You may use these files to transfer your projects
  around, but this requires some careful
  forethought.
• .mxd files do not store actual spatial data (only
  references to it). Therefore, simply copying a
  .mxd file to a thumb drive or CD is NOT going to
  transfer all of your work.
• References to spatial data may be stored as
  absolute paths (specific drive location on a
  computer) or relative paths (location of data
  files relative to the ArcMap Document - .mxd
  file). This option is accessible from the Map
  Properties option on the File menu of ArcMap.
  (The Data source options button)
• It is a good practice to create a new folder for
  each new map document. Store the .mxd at the
  base of the folder and the data in one or many
  sub-folders. If you do this, and store relative
  path names, it should be easy to transfer the
  project by copying the project folder to a new
  location. This approach is not always feasible,
  but it is a good approach to work towards.

18
Data Management

• About Layers and Layer Files (.lyr)
• In some cases it may be desirable to use the
  symbology and other modifications that you have
  made to a data set in multiple projects. These
  customizations can easily be shared among map
  documents by creating .lyr files.
• .lyr files allow you to save any of the
  modifications applied in the Layer Properties
  dialog of ArcMap in a stand along file.
• .lyr files are also a good tool for creating
  sub-sets of data without creating duplicate data
  sets. Like a .mxd file, the .lyr does not store
  a copy of the data, it just contains a reference
  to the source dataset. Because of this, the
  layer file always stays in sync with the source
  data set.
• If a map document containing .lyr files is to be
  moved to a different machine or onto removable
  media (thumb drive or CD), care must be taken to
  be sure that both the .lyr files and the source
  data are also transferred.