Using Geodatabase Replication To Manage Distributed Data

1
Using Geodatabase Replication To Manage
Distributed Data

• David BestRedwood National Park
• Craig Dalby
• Pacific West Regional Office

2008 GIS Data Management Conference
2
Contents

• The Problem
• Solutions and Consequences
• Geodatabase Replication
• Pilot Project
• Installing and Configuring SDE Services
• Managing Replication
• Results and Recommendations
• Next Steps

2008 GIS Data Management Conference
3
The Problem

• A great challenge is how parks will obtain and
  edit national GIS datasets.
• Options include moving data (and applications) to
  national servers, or
• Keeping data local, and pushing changes to
  national datasets

2008 GIS Data Management Conference
4
Centralized Data

• Field (remote) users give up local applications
  and local data in favor of Remote Desktop/Citrix,
  or
• Repeatedly download and upload data between local
  servers and national servers
• Few organizations have the network bandwidth to
  accommodate local applications directly editing
  remote data

2008 GIS Data Management Conference
5
Typical NPS Environment

• For most parks there is a dramatic performance
  improvement when working with local data.
• Local data makes it easier to customize user
  interfaces, more efficiently access other data
  stored on park servers, and integrate data of
  park-only interest.

2008 GIS Data Management Conference
6
Geodatabase Replication

• It is inconvenient to move data back and forth
  from the edit (local server) and integrated
  (national server) environments.
• ArcGIS 9.2 introduced new technology for this
  problem Distributed Geodatabase.
• Allows versioned SDE geodatabases to be deployed
  at multiple locations.
• Changes are synchronized with replica versions
  using either one-way or two-way replication.

2008 GIS Data Management Conference
7
Benefits

• A park can house and maintain data for a specific
  geographic area, gaining the performance of
  local data.
• Seamlessly, these data contribute to national
  datasets.
• Changes made at either the parent (EGIS) or park
  (child) can be replicated to the other.
• This allows parks and programs to retain
  stewardship responsibilities, while ensuring an
  authoritative national versions are maintained.

2008 GIS Data Management Conference
8
2008 GIS Data Management Conference
9
Need to Evaluate

• Geodatabase Replication is new want to test
  stability and reliability
• NPS computing environment is diverse want to
  test suitability in small parks with no IT
  support and large parks with dedicated GIS
  specialists and servers

2008 GIS Data Management Conference
10
2008 GIS Data Management Conference
11
Replication Options

• Conserve bandwidth by
• Re-use schema
• Where target already has data and only needs
  updating
• Schema only
• Mobile users who input only new data
• Register Existing Data
• Bypass data copy step and use existing data
• Replicate related data
• Include or not include relationship classes

2008 GIS Data Management Conference
12
About Synchronization

• Synchronization
• Synchronization is when one replica sends data
  changes and the relative replica receives
  changes.
• Changes include inserts, updates and deletes made
  to the replica version.
• Must be connected as the same database user that
  was used to create the replica
• Filters and Relationship Classes
• Filters and classes that were applied during
  creation of the replica are also applied during
  synchronization
• Data transfer based on exchanging replication
  messages
• Detect and automatically recover from errors by
  unreliable messaging.

2008 GIS Data Management Conference
13
PWRO Project Funds

• Participating Parks
• Redwood Lassen Volcanic
• Lava Beds Oregon Caves
• Sequoia-Kings Pacific West RO
• Had to overcome reluctance. Difficult for a park
  to see benefit.
• Common question what am I going to have to do?

2008 GIS Data Management Conference
14
Objectives
Can an out-of-the box standardized ArcSDE
deployment be seamlessly and consistently
deployed to park offices?

• Collect and enter building footprints
• Develop methods and procedures to install and
  configure two-way replication
• Implement two-way replication and train park
  staff
• Evaluate performance and make recommendations for
  EGIS use.

2008 GIS Data Management Conference
15
General Steps

1. Install SQL Server
2. Install SDE
3. Perform SDE post-install
4. Configure using Access Database
5. Create Replica on child
6. Schedule Synchronization

2008 GIS Data Management Conference
16
Configure Two-way Replication

• MS Access database with embedded VB code is used
  to automate
• Record information about location, IP addresses,
  server name, database locations
• Create new Active Directory groups for
  administration, editing, and viewing park-based
  EGIS data
• Create folders/shares for administrative files
• Create SDE connection files
• Enable SQL backup
• Implement SQL database roles and AD security

2008 GIS Data Management Conference
17
2008 GIS Data Management Conference
18
2008 GIS Data Management Conference
19
2008 GIS Data Management Conference
20
Database Produces Consistent Share Name with AD
Security
OTHER PARKS
Lassen Volcanic
Redwood
PWRO
EGIS
Lava Beds
21
(No Transcript)
22
Create database EGIS (SDE created by SDE install)

• Create logins for
• Administrators
• Editors
• Viewers

23

• Set permissions
• for SDE and EGIS databases
• Create Database Roles
• Editors
• Viewers

24
Connects to parks version on the EGIS database as
the LABE SDE user on the EGIS server
EGIS EGIS Database Park Version as
park SDE use PARK EGIS Database Default
Version as park SDE user
Connects to EGIS database as the LABE SDE user on
the parks SDE server
25
SDE Connection Files Can Be Read By ArcCatalog
2008 GIS Data Management Conference
26
Right-click to viewConnectionProperties
27
Connection Properties
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Automated Backup of SQL Databases Are Consistent
Lassen Volcanic
Lava Beds
PWRO
OTHER PARKS
32
(No Transcript)
33
Python Script for Synchronization
2008 GIS Data Management Conference
34
Create Replica on Child

• In most cases an EGIS responsibility
• Data is mainly pulled instead of pushed.
• No need to disclose EGIS security to parks.
• ArcMap provides spatial filter to limit replica
  to parks geographic area (not available in
  ArcCatalog)
• Filters are great way to eliminate versions and
  posting

2008 GIS Data Management Conference
35
Activate Distributed Geodatabase Toolbar
36
Load data that will participate in replica into
map
37
Specify Type of Replica
38
Specify Connection and Replica Name
REPLICA OWNER PARENT CHILD
39
Specify Spatial Extent
Use to limit geographic area where
synchronization occurs. Useful for avoiding SDE
versions
Use for trusted editors and offices with
oversight responsibility
40
Managing Replication

• Synchronization can be manual or automated.
• Synchronization begins for each park as a
  separate task within Schedule Tasks.
• The task is a batch (.bat) file that calls the
  python script created earlier.

FROM EGIS TO
2008 GIS Data Management Conference
41
Monitoring Replication

• Right click SDE database
• ? Distributed Geodatabase ?Manage Replicas

2008 GIS Data Management Conference
42
Replication Manager
Replication Log
Viewing the Replication Log
43
Results and Recommendations

• Complete install about 3 hrs/server
• Get Administrator access
• Other Flavors of SQL/SDE probably not worth coast
  savings
• SQL backup very easy and reliable
• SDE Service needs monitoring
• Two-way replication avoids permission management
  with versions
• Replica design should be updatable
• Infrastructure mostly adequate

2008 GIS Data Management Conference
44
Results and Recommendations

• Have not trained park users, but do not expect
  difficulties.
• Unforeseen benefit may be rapid-deployment of
  region or network-based integrated datasets.
• Datastore should consider one-way data
  subscription service for national datasets not
  edited by parks.
• Annotation is an easy and very useful national
  test dataset.

2008 GIS Data Management Conference
45
Next Steps

• Complete Klamath Network Parks.
• Encourage frequent park editing.
• Review by EGIS Team.
• Add new theme.
• Move parent to RIMD.
• Work on VPN solution.
• ELA for SQL Server.
• New participants?

2008 GIS Data Management Conference