bottlenecks for large TANGO systems

1
bottlenecks for large TANGO systems

• Jean-Michel Chaize
• European Synchrotron Radiation Facility (ESRF)

2
Steady state situation
Polling event widely distributed
Clients
Clients
Clients
Clients
Clients
Clients
Clients
Clients
Db server
Powercut
Server
Server
Server
Server
Server
Server
Server
3
Powercut situation
Clients
Clients
Clients
Clients
Clients
Clients
Clients
Clients
DBImportDevice
Db server
Power back
Server
Server
Server
Server
Server
Server
Server
4
Restarting from powercut
Clients
Clients
Clients
Clients
Clients
Clients
Clients
Clients
DBImportDevice
Export devices Get properties Get attribute
properties
Db server
Server
Server
Server
Server
Server
Server
Server
5
Work-around

• Enlarge the maximum number of open connection
• Increase the number of threads
• Slowdown the import devices
• Distribute the load over several database servers

Need a manual tuning Not adequate for long term
6
Find more efficient solutions

• Heavy database access during server startup(From
  E. Taurel March 2007)
• 10 3classnum (3 natt)devnum read access
• 20 calls for a simple server exporting a device
  with 4 attributes
• 46 calls for a 2 class server exporting 5
  devices.
• 1 classnum devnum write access.
• At esrf when restarting the system
• 15000 read and 3000 write on the db server
• Proposal
• Review ds startup and use Stored procedures (E.
  Taurel March 2007)
• Potential gain of factor 10.

7
Find more efficient solutions

• Heavy database access due to clients on missing
  devices
• Each client retry periodically to import each
  device
• Proposals- Distribute the naming service. (JL
  Pons sept 2006)
• - Use MySQL 5 stored procedure to avoid the SQL
  query compilation time.
• - cache IOR in client API to allow re-import
  without database
• - Use an event to notify the re-export of a
  missing device.

8
A proposal for a distributed naming service (JL
PONS)

• - Allows fast reconnection - Avoids the large
  number of DbImportDevice calls on the database.-
  The major difficulty will be to manage the
  backward compatibility with old server that wont
  be exported in the name server.

ImportDevice (Fisrt call)
Client
ImportDevice
DB Server
Name server (Fixed port)
Server
ExportDevice
Name
IOR
Server
ExportDevice
Server Host
9
Summary of proposals

• Use stored procedures in dbserver new server
  api
• A factor of 10 on the number of call on server
  startup
• Cache IOR information in client API
• Minimize overload due to client imports
• Use events when re-exporting devices
• More complex implementation, large gain.
• Implement a distributed naming service.
• Complex implementation, large gain.