An Extensible

1
An Extensible Automated World
VSX, Shell, GAX, DSL, Factoriesoh my!!

• Santosh Benjamin

santosh.benjamin_at_yahoo.com http//santoshbenjamin.
wordpress.com
2
Agenda

• VSX 101
• Guidance Automation
• Domain Specific Languages
• Software Factories
• Roadmap
• Resources

3
VSX Am I Bothered?

• IDE
• Vast, feature rich
• A long journey from the old days (pre VS 6.0)
• Extensibility is crucial
• Not possible to fulfil every single developer
  need
• For Developers
• Extensibility allows you to fit the tool (IDE) to
  the process
• For ISVs
• Extensibility allows you to make the tool (IDE)
  your platform
• therefore
• We should be bothered !!!

(The Catherine Tate Show)
4
VSX Options
Developer Reach
5
Visual Studio Templates

• Project Item templates
• Reusable and customizable project and item stubs
• New Project and Add New Item dialog boxes
• Can be used to provide starter kits
• Accelerate the development process
• Easy deployment just a .zip file
• Content files source code files, embedded
  resources, project files, ...
• Metadata stored in a .vstemplate file (XML)
• Copy to a predefined folder (local or on the
  network)
• DEMO
• Out of the box templates locations
• Creating and customizing a template
• Limitations
• Capabilities are limited (templates are dumb)
• Can be extended using IWizard interface to make
  them a little more dynamic

6
The VS Automation Model

• Automation object model contains gt140 objects,
  allowing control of the IDE from a solution level
  all the way to a single parameter of a single
  function
• The Legacy
• Old components never die
• they only get wrapped in layers and buried deep
• The COM has not gone away ?
• In the old days the common interface was IUnknown
  (and for most of us it is still not known ? )

7
Macros

• Macros can leverage nearly all of the features of
  the .NET Framework, as well as most of the
  automation model
• Macros accessed via Tools Macros
• Macros have some limitations
• Macros can only be written in VB
• Only source code distribution no binaries
• Macros cannot be used to implement new tool
  windows, commands, Tools Options pages

8
Add-Ins

• Add-Ins also leverage the automation model.
• Add-ins can implement new tool windows, commands,
  Tools Options pages
• Seamless addition into the IDE
• Choice of language (VB/ C / C)
• Simple registration using XML ( no more COM
  registration )
• However, Add-ins cannot implement new document
  types, new project types, new debug engines, etc.
  which require a package

9
Wizards

• In VS, wizards generally ask a series of
  questions that solicit input from a user, and
  then use the results to generate code.
• Do not always have UI. Can be programmed to
  invisibly generate code behind the scenes.
• 3 Types
• New Project wizards to generate new code for a
  particular type of project,.
• Add New Item wizards to add new items, such as
  Web forms, text files, HTML pages, XML pages, and
  so forth, to a project.
• Custom wizards Called directly from add-ins,
  macros, or other types of code. They may or may
  not display a UI. In either case, they generate
  code. This type of wizard is used the least
  often.
• IDTWizard the original one. Implemented using
  vsz files, HTML and script code
• IWizard new kid on the block in VS05 and VS08.
  C and VB project systems primarily use this.
  VSTemplates are usually extended using IWizard.

10
Packages

• Some users need more power and flexibility than
  what they get via macros, add-ins and wizards.
  They may need custom editors and so on.
• To achieve this level of integration, we need to
  write packages
• Actually the VB, VC, VC project systems are
  implemented as packages on top of the basic IDE
  shell.
• VS Packages are created using the Visual Studio
  SDK in any language

11
Package Extensibility Points

• IDE
• Project systems, Languages, Editors, Tool
  Windows,
• .NET Designers, Menus Commands, Toolbox
  Integration, Tools Options Page, Property Browser
• Help Integration, Setup and Deployment
• Debugger
• Custom Debug Engine, Expression Evaluators , Type
  Visualizers
• Source Control
• Project systems enabled to participate in source
  control
• Source Control Provider ( Plug-in or Package )
• Data Sources
• Design-time Data Source Drag-n-Drop
• Data Source Provider
• . and more

12
Visual Studio SDK

• A developer suite, framework /platform that
  allows you to achiever finer control and deeper
  access into the IDE
• With this, we now get access to the same
  interfaces and internals of the platform as the
  teams who write the IDE itself.
• Key Concepts
• Experimental Hive
• The SDK creates the separate registry hive to do
  prototyping work
• Samples auto-register in Experimental Hive
• The new SDK enables a reset all for the
  experimental hive
• DLK and PLK
• Developers use the Developer License Key,
  provided with the SDK
• Shipping requires Package Load Key
• If you distribute your package to other
  developers without a PLK, then they need to have
  the SDK to run it.
• Setup and Deployment
• Final product needs to author a setup and
  deployment project that handles
• Presenting your company EULA

13
Package Architecture
Your Native Package
Your Managed Package
or
MPF 8.0
Interop Assemblies
Native Interfaces
Visual Studio 2005

• SDK Provides
• Native Interface IDLs
• Interop Assemblies
• Samples, Doc, templates, etc
• Package Wizard

• Visual Studio 2005 is the platform
• Native Interfaces provide extensibility points
• Interop Assemblies enable managed code access

14
The Visual Studio Shell

• A streamlined IDE on which you can build tools
  etc for specific markets (like Eclipse).
• The shell has always been there (in VS2005 it was
  referred to as Premier Partner Edition) but not
  as a separate product with its own identity as it
  is now.
• Integrated Mode
• Optimized for programming languages (e.g.) Iron
  Python)
• Applications built on the integrated Shell will
  automatically merge with any other editions of
  Visual Studio installed on the same machine.
• Isolated Mode
• Optimized for specialised tools (e.g.)
  Storyboard Designer
• Applications built with the isolated Shell will
  run side-by-side with any other editions of
  Visual Studio installed on the same machine.

15
Extensibility Map - Summary
Type Benefits Limitations
Customization IDE / VS Templates Simple Cannot control IDE, can only customize
Macros Full control of IDE leverage automation model and framework Cannot add features Provided in source Only written in VB
Add-ins Full control of IDE Compiled for performance and protection of source Simple registration Can implement new tool windows, commands, Tools Options pages Cannot implement new document types, new project types, new debug engines,
Packages Full control of IDE Compiled for performance and protection of source Full ability to add new features Complex development model
16
VSX In Action - Demos

• Macro
• Add-In
• Add-In with Wizard
• SDK Tool Window

17
Agenda

• VSX 101
• Guidance Automation
• Domain Specific Languages
• Software Factories
• Roadmap
• Resources

18
Guidance Automation 101 Whats the big deal?

• We all have good intentions
• We want a consistent way of developing software
• We want our teams to share and follow standards
  and good practices
• So whats the problem?

Doh! Is that the architecture standards document?
!!!
Size is only 1 problem. Arguably the more
pressing problem is that its passive. And we all
know how enthusiastic we are about reading
through tons of info to get to the part we want
(especially if it was written by someone else ? )
Pages and pages of guidelines
What we need is a way to make all this carefully
(sometimes painstakingly) collected material
available at the point of need and even better,
make it actually executable.
19
Enter ? Guidance Automation

• The Aim
• Make reusable code and pattern assets available
  in Visual Studio
• Integrate reusable code into applications
• Guide developers through complex procedures
• Built and used by Microsoft patterns practices
• Built upon Visual Studio SDK
• Used extensively in their various Software
  Factories

20
Guidance Life Cycle
Only the package author needs GAT. The package
user (usually a developer) only needs GAX to run
the package and produce source code. The end
product of all this is source code. The end user
only sees a web app or winforms app and doesnt
care how it was built. There are NO runtime
library requirements for the end user.
21
GAT The Key Elements
Recipes Recipes automate activities that developers would usually perform manually, often by following a series of instructions.
Actions Actions are atomic units of work called in a defined sequence by recipes. The sequence is specified in the recipe definition.
T4 templates A Text Template Transformation Toolkit (T4) template consists of a combination of text and scriptlets (expressions in VB/C evaluated at runtime)
Wizards Wizards are value gathering strategies used to gather values from recipe arguments. .
Type Converters Type converters validate the value of a field and convert them from their user interface representation to a type representation
VS Templates GAT allows you to associate these with recipes to make solution/project/item generation very parameterised
22
Demo GAT In Action

• Using the Web Service Software Factory (Dec 2006
  Refresh)
• Inspecting the internals of the Service Factory

23
Building A Guidance Package

• So how would you build one of those things?
• A Guidance Package is a VS solution with a
  specific layout and some specific artefacts in
  it. It is not trivial to build and easy to get
  wrong.
• To make things easier we have a Meta Package
  a package package

24
Building a GP - Demo

• Using the out of the box Meta Package
• Using the Clarius Meta-factory (part of Software
  Factories Toolkit)
• Enhances the VS out of the box GP with an
  enhanced structure compilation settings,
• more context driven recipes and visual elements
• Some advanced techniques such as GAX Scripting
• Provides a much better authoring experience

25
VSX, SDK and GAT

• VSX (vs.) GAX
• Without VSX there would be no GAT
• GAX/GAT is a vertical application of VSX for a
  specific purpose
• With VSX you can build tools products that have
  nothing to do with GAX/GAT.. but then while
  building the tools GAT can help!!
• GAT meets the SDK
• SDK has a steep learning curve
• GAX driven tooling allows a streamlined approach
  to developing with the SDK
• Examples
• VSSDKAssist Factory allows us to create,
  configure, deploy and test various items such as
  Commands, Services, ToolWindows, Custom projects
  etc
• VSShellAssist Factory allows us to create,
  configure and test shell setup projects, load
  keys, splash screens etc

26
Some Best Practices for GAT

• Use GAT when
• You have a specific problem with a specific
  solution that
• Is automatable
• Must be executed multiple times
• Has a development time or risk that is too high
  to do manually
• You cant use dumb Visual Studio Templates
• You want to prepare for building a Software
  Factory
• Do not use GAT when
• You will use the Guidance Package only once
  ever
• You cant/dont want to invest time in the
  learning curve
• You just think its cool

Jelle Druyts Deep Dive into GAT
27
Authoring Guidance Packages

• There is a learning curve
• Do not involve everyone
• Make 1 person/team responsible for GAT
• Once the knowledge is there, investment pays off
  quickly
• Dont try to build everything at once
• Start automating simple scenarios
• Incrementally increase complexity
• Add templates, logic and guidance along the way
• Borrow / Reuse / steal !!
• Dont reinvent the wheel
• Most of the factories have source code and thats
  available for non commercial reuse in your own
  stuff (check the license carefully)
• At the very least you can see how someone else
  has implemented a similar idea to yours.
• The Guidance Extensions Library (GEL) contains a
  collection of recipes, actions etc used by PP
  Factories and made available for us to use

Jelle Druyts Deep Dive into GAT
28
Agenda

• VSX 101
• Guidance Automation
• Domain Specific Languages
• Software Factories
• Roadmap
• Resources

29
Domain Specific Languages

• Domain-Specific Languages (DSLs) typically are
  small, highly focused languages used to model and
  solve some clearly identifiable problems which an
  analyst, architect, developer, tester, or system
  administrator must handle.
• Some examples of traditional DSLs
• SQL for data manipulation,
• XSD for XML document-structure definition, etc
• How does it apply to us?
• Most teams can talk about their software in very
  team specific lingo using specific names for
  recurring types of components, patterns.
• The area the team works in is their domain and
  the terminology forms their language.
• There is now technology available in this area
  for us to build on.

30
DSLs The Visual Element

• Just being able to use a DSL will help teams get
  more consistency in system design and
  implementation.
• A DSL does not have to be visual. However, being
  able to put a face to these things (if
  possible) helps in understanding and
  communicating the system better
• Some teams use custom Visio stencils and some go
  a step further and use the Visio automation model
  to generate code from the stencils, generate
  documentation and so on.
• But there is no common framework around Visio
  code generation nor a platform to make things
  easier
• This is where DSL Tools for Visual Studio comes
  in

31
DSL Tools For VS

• Domain-Specific Language Tools allow Visual
  Studio 2005 developers to create their own
  graphical designers and code generation tools
  like the ones you find in Visual Studio today,
  such as the Class Designer, Application Designer
  etc.
• Using DSL Tools, designers can
• specify the domain model (semantics) for a new
  language
• create a custom graphical editor that uses a
  domain-specific diagram notation (syntax) and
• construct custom text templates that use models
  created using the graphical editor, to generate
  source code and other files.

32
DSLs examples(1)

• Service Factory Modelling Edition
• Service Contract Model, Data Contract Model, Host
  Model

33
DSL examples(2)

• VS Team Architect
• Application Designer , Logical Data Center
  Designer
• VS Team Developer
• Class Designer
• Biztalk
• Orchestration Designer, Pipeline Component
  Designer
• WF
• Workflow Designer

34
DSL vs UML

• UML is very general purpose. It can be considered
  as a DSL but the domain would be the whole of
  software development.
• The focus of a typical DSL is much narrower. It
  picks on specific verticals and (sometimes)
  specific types of applications in that vertical
• Some might argue that you could achieve similar
  results with UML Stereotypes but thats quite
  hard and tooling is not easily available to
  translate those stereotyped models into your
  specific code AND back again
• Steve Cook , a leading DSL proponent, was on the
  IBM board that worked on the standardisation of
  UML so he is definitely an authority on what it
  can and cant do
• However UML hasnt gone away and it isnt going
  away.
• The Class Designer extends the familiar UML
  notation to accommodate things that are unique to
  .NET classes (attributes etc). Some new tools in
  Rosario will allow architects to reverse
  engineer code into sequence diagrams and other
  visualizations that will have many UML elements
  in it.

35
Agenda

• VSX 101
• Guidance Automation
• Domain Specific Languages
• Software Factories
• Roadmap
• Resources

36
What is a Software Factory?
The easy definition (PnP) A software factory
is a structured collection of related software
assets. When a software factory is installed in a
development environment, it helps architects and
developers predictably and efficiently create
high-quality instances of specific types of
applications.
The Official Definition

• A Software Factory is a software product line
  that configures extensible tools, processes and
  content using a software factory template based
  on a software factory schema to automate the
  development and maintenance of variants of an
  archetypical product by adapting, assembling and
  configuring framework-based components.
  
• Jack Greenfield, Keith Short, Software Factories

37
In more concrete terms.

• A Tool, to automate the assembly and
  configuration of solutions addressing a specific,
  understood problem domain
• Runs within, and integrates with, Microsoft
  Visual Studio
• Uses models, designers, editors, wizards, etc.,
  to provide abstractions of the solution domain
• Outputs mostly generated solution (product) that
  may require manual customizing to complete
• Also contains executables, readables, reusables.

Build Your Own Software Factories Jezz
Santos, Edward Bakker
38
What would we build it with ?
Asset Tool/Technology
Automated Action GAX/GAT recipe, with wizard
Code/artifact Templates Text Templating Engine
Project or Solution Template Visual Studio template with GAX/GAT recipes
Domain Specific Language DSL Toolkit
Logical Application System Design Visual Studio Team Edition for Architects (SDM SDK)

• GAX/GAT available from patterns practices
  All other tools available in Visual Studio SDK

Build Your Own Software Factories Jezz
Santos, Edward Bakker
39
Its like an umbrella

• This SF approach brings in all the good stuff
• Patterns
• Guides
• Guidance Automation
• Domain Specific Languages
• ..and puts them all under one roof

40
Demo

• Service Factory Modelling Edition
• Service Contract Model
• Data Contract Model
• Host Model

41
Factories.. Things to remember

• Solutions that are going to be modelled in the
  factory should be built before the factory itself
  is built.
• Keep in mind that the factory-building process
  will take two to three times longer than a
  one-off product such as building a Web service
  from scratch.
• Finally, keep in mind that ROI will not come
  until the factory has been put to use at least
  three times.

Build Your Own Software Factories Jezz
Santos, Edward Bakker
42
Demo . outcome

• So what have we seen?
• The DSL approach provides a higher level of
  abstraction
• This itself gives the Service Factory more value
  than its previous incarnation which could be
  considered as code generation on steroids.

43
Factories. Just the beginning

• What we have seen isnt all there is to software
  factories in fact its just the beginning
• The PP software factories (esp Service Factory)
  are now moving in the right direction with the
  DSL abstraction over code generation and the
  in-context guidance
• But in reality we are only scratching the surface
  of Software Factories as envisioned by Jack
  Greenfield and others.
• There are advanced things like the Software
  Factory Schema , Software Product Lines and so
  on.

44
Any feelings of déjà vu?

• A common criticism is that this like the 80s all
  over again with respect to CASE Tools
• As with UML , the key players here have been
  there, done that.
• Software Factories now takes a more modular
  approach than the top down process heavy approach
  that was seen with the CASE tools. You can pick
  and choose what suits your development approach
  best and the tooling does NOT try to dictate the
  process.
• Examples Even in the area of layered VS
  solutions, the WSSF allows you to have a single
  project for everything if you dont want 8
  separate assemblies. Again, you can use just the
  service interface models and do everything else
  in your own way , or just use the Repository
  Factory to control the data layer and do the rest
  yourself.
• So no, its not CASE all over again.
• But if youre a die-hard cynic , perhaps it would
  be best to wait and watch!!

45
Agenda

• VSX 101
• Guidance Automation
• Domain Specific Languages
• Software Factories
• Roadmap
• Resources

46
Roadmap

• Visual Studio ( Team System)
• In Rosario, Hawaii and beyond , Factories are
  going to be baked into the product.
• Edward Bakker has some posts on how this
  manifests itself in Rosario
• Team Factories from the initiation of the
  project on TFS, through to work item allocation,
  code generation, testing and maintenance.
• Service Factory
• Guidance to encompass Biztalk, WCF, WF, Astoria
  etc (will start as documented guidance and may
  eventually become full fledged factories)
• REST RSS capabilities
• Leverage Metadata repositories

47
Agenda

• VSX 101
• Guidance Automation
• Domain Specific Languages
• Software Factories
• Roadmap
• Resources

48
Resources (1)

• Books
• Software Factories (2004) - Jack Greenfield,
  Keith Short
• Working With Visual Studio 2005 Johnson, Skibo,
  Young
• Professional Visual Studio Extensibility (2008) -
  Keyvan Nayeri
• VSX Articles
• VS Templates and Starter Kits
• Developing Visual Studio Wizards (IWizard)
• GAX/GAT Articles
• Introduction to GAT (MSDN)
• Guidance Automation Series (Jelle Druyts)
• Blogs (Hall Of Fame)
• VSX Carlos Quintero
• SF Jezz Santos , Edward Bakker , Jelle Druyts ,
• Clarius Blogs Pablo Galiano, Victor Garcia,
  Daniel Cazzulino
• Newsgroups
• MSDN Forums (VSX, GAT)

49
Resources (2)

• Webcasts Videos
• Deep Dive into GAT Jelle Druyts MSDN Belux
• Packaging Design and Architecture Guidance
• Introduction to Service Factory
• Build Your Own Software Factory (EMEA Spotlight
  TechEd 07)
• Visual Studio DSL Tools Stuart Kent
• Software
• GAX (Feb 08) and GAT (Feb 08)
• Factories
• Web Service SF , Web Client SF, Repository
  Factory
• Smart Client SF, Ent Lib 3.1 Application Block
  Software Factory
• Biztalk Solution Factory
• Misc
• My blog starting a series on GAT shortly

50
Summary

• Visual Studio is highly extensible and can now
  justifiably be considered more than just an IDE ,
  it is a tools platform. This is a good time to
  start getting into the field of VS extensibility.
  
• Guidance Automation leverages the extensibility
  of VS to allow us to
• Leverage common best practices in an executable
  form
• provide active guidance to development teams
• DSL Tools provide us a platform to build visual
  languages and link into code generation etc
• Software Factories bring everything together to
  help us along the road to a higher level of
  consistency and accuracy in software development.

51
Q A