Life Beyond Distributed Transactions An Apostates Opinion

1
Life Beyond Distributed TransactionsAn
Apostates Opinion

• By Pat Helland
• Amazon.Com
• Jan 8th, 2007

Apostate noun One who renounces a previously
held belief.
Todays Goal Offer hopefully insightful opinions
about scaleable apps.
2
Assumptions(Dont Gotta Prove These Just Plain
Believe Then)

• Grown-Ups Dont Use Distributed Transactions
• The apps using distributed transactions become
  too fragile
• Lets just consider local transactions.
• ? Multiple Disjoint Scopes of Serializability

• Want Almost-Infinite Scaling
• More of everything Year by year, bigger and
  bigger
• If it fits on your machines, multiply by 10, if
  that fits, multiply by 1000
• Strive to scale almost linearly (N log N for
  some big log).

3
Uniquely Keyed Entities

• Not All Data May Be in a Single Transaction
• We Must Collect the Data into Pieces
• We Must Annotate the Boundaries of the Data
  Guaranteed to Be Transactional
• Must Remain Transactional Even If We Repartition!
• An Entity
• A Collection of Data that Fits on a Single
  Machine
• Identified by a Unique Key
• Assume the Scale-Aware-Code Never Partitions an
  Entity
• The Unique Key Defines the Data that Cant Be
  Partitioned

4
Transactions and Entities

• A Transaction May Update a Single Entity
• The Scale-Aware-Code (and API) Guarantee It
• The Entity Is Never Partitioned
• A Transaction Must Not Ever Update Two Entities
• Even If the Two Live on One Machine Today
• Tomorrow, They May Repartition to Different
  Machines

5
Repartitioning and Entities

• Entities Allow Scaling
• Entities Remain Intact Even when Repartitioning
• The Application Can Count on the Integral Nature
  of Each Entity
• It Is OK to Know that the Entire Entity Is Local
• It Is OK to Work on Anything in the Entity at Once

Entity JKL
Entity FXQ
Entity ABC
Entity ABZ
Entity NAO
Entity MOE
Entity GHI
No Promisethat TwoDifferentEntities Stayon
the SameMachine!!
Entity DEF
Entity LMN
Entity RST
Entity RST
Entity JKL
Entity RAA
Entity JAA
Entity EFG
Entity XYZ
Entity LMN
Entity XYZ
Entity GHI
Entity XYZ
Entity LMN
Entity KZU
Entity FAW
6
Thinking about Alternate Indices

• Entities Must Have a Unique Key
• Unless the You Begin with the Same Key, You
  Arent the Same
• CANNOT Guarantee the Alternate Index Will
  Co-locate with the Entitys Primary Key
• By Definition, Alternate Indices Dont Have the
  Same Key!
• We Must Index Them with a Different Key
• Alternate Indices CANNOT Be Updated in the Same
  Transaction as the Primary Data
• There Is No Way to Guarantee They Are on the Same
  Machine
• They Must Be Updated in Different Transactions

7
Entities Are Connected by Messaging

• Entities Are Key-Named Boundaries for
  Transactional Work
• Transactions Never Span Entities
• The Scale-Aware-Code May Move Them to Repartition
• The Only Way to Communicate across Entities Is
  with Messaging!
• The Scale-Aware-Code Is Responsible for Finding
  the Correct Entity (by Key-Name) and for Routing
  the Message to It

Messaging Is in Quotes Work Is Invoked --
Potentially across Machines -- Definitely across
Transactions!
8
Messages Connect Entities

• Messages Are the Only Way into and out of
  Entities
• They Are Produced by Transactions
• They Are Consumed by Transactions
• Transactions Are Local to the Entity

9
Entities Connected by Partnerships

• Mostly, Messaging Occurs between Two Partner
  Entities
• Usually, a Two-Way Exchange Moving Both Entities
  State
• Each Keeps Data about How Far Its State Has
  Advanced

10
Tracking a Partner with Activities

• Activity Refers to the Knowledge about a Partner
  Entity
• Descriptions of What Messages Have Been Received
• Descriptions of What Obligations Exist to the
  Partner
• The Foundation for Workflow to Replace
  Distributed Transactions
• Two Basic Observations Wrapped Up in the
  Activity Concept
• Work Across Entities Is Workflow Based on
  Two-Party Relationships
• The Granularity of the Workflow Participant Is an
  Entity (Fine-Grained)

Entity-X
Entity-Y
Entity-W
Entity-Z
11
Vocabulary and Assertions
New Vocabularyfor Discussing Scale
Assertionsabout Large Scale Apps