Vod:Pod

1
VodPod

• Hyperaggregation and
• social recommendation for video

Scott Persinger, Head of Engineering
2
Who We Are

• Mark Hall former VP International, Real
  Networks
• Scott Persinger former VPE Gofish.com
• Spencer Miles founder of JotDot.com and
  Showaholic.com
• Located in downtown SF
• Started June 1, 2006
• Site launched Dec 2006

3
What is vodpod?

• Video aggregation and recommendation site
• Many inputs (direct upload, embed codes)
• Many outputs (site, widgets, rss)
• Social network model
• Content is organized in pods
• Pods are based on a theme or just a social group
• Taste-maker model
• Show people the coolest videos Ive found
• See what are my friends watching
• Its a delicious web 2.0 stew
• UGC, folksonomy, ajax, vitality and attention data

4
(No Transcript)
5
Rails Triumphs

• Conception to deployment 6 months with 2
  engineers
• More features than my last application that
  site built in 6 months with 10 engineers using
  Java
• Simplicity scales, both conceptually and for
  performance

6
Rails Favorite Bits

• Migrations!
• ActiveRecord works as expected 95 of the time
  compare to Hibernate (40?)
• Ajax support with Prototype Scriptaculous
• Plugins and community
• Implemented ferret search in 4 hours

7
Rails Sweetness

• RJS Dynamic Script Tags
• Cross-domain Ajax to search.vodpod.com
• render(update) page page.replace_html
  id, partial gt webresults
• Hpricot Mechanize
• Super easy DOM parsing and web automation
• IO.popen_timeout
• Limited execution popen, kills process after X
  seconds

8
Rails Annoyances

• Where do I put business logic?
• Wait, you modified the Fixnum class?
• Would like better support for page components
• IDE lacking, no real debugger
• Poor support for background tasks
• Some poor library support
• http-access2 sucks

9
Challenge AR limitations

• We dont delete db records, we mark with
  is_deleted
• Counter cache used all over, but only works with
  real record deletion
• Gets worse when you create relations with more
  conditions
• User.videos_count depends on both
  videos.is_published and videos.is_deleted
• Solution remix_belongs_to
• belongs_to façade adds logic to keep
  counter_cache correct when relation has
  conditions
• Uses brute-force recalc

10
Challenge Scaling

• Shared-nothing, right?
• Doh! What about all the files on the filesystem?
• Uploaded photos
• Uploaded videos
• Ferret search indexes

11
Scaling File assets in the cluster

• Video files
• served from the specific host
• Images
• replicated on demand between servers
• Ferret index
• Unique search server to ensure consistency
• Yuck, too much complexity

12
Better Approach?

• Shared network storage
• NFS? Samba? Eeewww
• Performance and reliablity suspect
• NAS very expensive
• Amazon S3
• Likely the best solution, give them the headache
• But how to integrate it easily?
• Bandwidth is expensive

13
The Pitch

• Come over to vodpod.com!
• Collect your favorite videos and publish to your
  blog.
• Were hiring!

scott_at_remixation.com