Desktop Grids

1
Desktop Grids

• Jun Ni
• Department of Computer Science
• The University of Iowa

2
Introduction

• Reinvent its existing PC infrastructure as an
  enterprise-class computing resource to provide
  significant additional capacity for
  compute-intensive applications without adding
  significant additional overhead costs.

3
History

• Many people are first exposed to the idea of the
  aggregation of PC processing power through one of
  the many computing
• Some examples
• (SETI_at_home, http//setiathome.ssl.berkeley.edu/),
  evaluating AIDS drug candidates, (FightAIDS_at_Home,
  http//www.fightaidsathome.org/)
• Screening for extremely large prime numbers
  (Greater Internet Mersenne Prime Search,
  http//www.mersenne.org/prime.htm)
• Predicting climate on a global scale
  (ClimatePrediction.net, http//www.climatepredicti
  on.net/index.php).

4
Paradigm

• All of these projects are based on the idea of
  enrolling, i.e., a conscious decision on the part
  of a PC owner to sign up with a particular
  organization to allow the spare computational
  cycles of his PC to be used by the selected
  project.

5
Mechanism

• Upon enrollment, a small control program is
  downloaded to the PC.
• This program is responsible for communicating
  with the central project server (using the public
  Internet connection of the PC) as well as
  harvesting the spare capacity of the machine by
  executing cause-related computations relayed by
  the central server.
• Typically, these projects use relatively short
  communication packets to drive comparatively long
  computations on the enrolled PC.
• This is an attempt to be minimally intrusive on
  the user and his Internet connection.

6
Mechanism

• The method for consuming the spare capacity of
  the PC can be as simple as executing the
  cause-related computation in the place of the
  normal screensaver (taking advantage of those
  instances when the computer is completely unused)
  or as complex as executing the cause-related
  computation continuously as an idle-priority task
  within the Windows environment (giving preference
  to any user-initiated tasks and then soaking up
  any remaining capacity).

7
Recourses

• These ad hoc collections of work-based and
  home-based PCs from around the world are an
  example of PC-based distributed computing and
  serve as the forerunners of todays true Desktop
  Grids.

8
Key Issues

• Some of the key Issues that arose out of these
  projects include
• Resource ManagementAll Internet-based grids use
  passive resource management
• rely on the enrolled PCs to initiate
  communication with the central administration
  server on a periodic basis.

9
Key Issues

• limits the degree to which the timeliness of
  results from such a grid can be predicted.
• limits the ability to re-prioritize the
  computational behavior of the grid (for example,
  replacing the PC that is working on a particular
  task) in a timely manner.

10
Key Issues

• Communication and Data SecurityHTTP is the
  communication protocol between the PCs and the
  central server.
• Even if some form of encryption is used in
  transit, the data usually reside in an
  unencrypted format on the enrolled PC.
• This limits the nature of the problems that can
  be attempted over the public Internet to those in
  which compromise of the data is not a pressing
  issue.
• In addition, in some cases, the answers produced
  on the enrolled PC may be vulnerable to
  tampering, causing the confidence in the results
  to be lower than desired.

11
Key Issues

• Machine HeterogeneityA wide variety of machines
  might be enrolled these can vary in CPU speed,
  RAM, hard-drive capacity, and operating system
  level.
• The management infrastructure either needs to
  operate at the lowest common denominator or
  needs to be aware of differences in the machines
  and assign tasks appropriately.

12
Key Issues

• Resource AvailabilityThe entire cause-computing
  paradigm relies on the idea of voluntary
  participation.
• As such, the availability and utility of any
  particular resource is subject to the whim of the
  person controlling the PC.
• The PC may be turned off for the night, the
  screensaver may be changed, the control program
  may be disabled (either deliberately or
  inadvertently), etc. This adds another layer of
  unpredictability to the performance expectations
  that can be associated with such a grid. (fault
  Tolerance)

13
PC Distributed Computing

• Distributed computing in the corporate world
  evolved out of the high-performance computing
  grids consisting of inter-networked UNIX and/or
  Linux machines.
• Corporate users in that environment had come to
  expect resource management, security, and
  availability as an inherent part of their
  distributed computing infrastructure.
• Many corporate users realized that the
  aggregated, unused power of the desktop/laptop
  PCs assigned to employees represented a large
  pool of computational cycles that were being
  wasted and not benefiting their company.
• However, concerns related to the four concepts
  discussed above limited the interest and
  utilization of PC-based distributed computing
  using the Internet paradigm among enterprise
  users.

14
PC Distributed Computing

• In addition, this paradigm did not acknowledge
  (or attempt to exploit) the fundamental
  differences between a collection of PCs connected
  by a corporate intranet and an ad hoc assortment
  of typical home-based PCs connected using the
  public Internet.

15
PC Distributed Computing

• These differences include
• Networks
• Network ConnectivityMany corporations have their
  PCs on dedicated high speed (0.1 Gbps) or
  very-high speed (1Gbps) networks, or 10 Gbps
  networks.
• A faster, dedicated network connection allows
  much more freedom in designing a distributed
  application
• However, some of these machines may have only
  intermittent or occasional connection to the
  corporate network (and when they are connected,
  it may be through a much lower bandwidth pipe).
• Because many organizations are using portable
  computers as the primary desktop computing
  device, both the duration and the quality of any
  devices connection with the corporate network
  are difficult to predict.

16
PC Distributed Computing

• Required Participation
• Participating in a distributed computing effort
  can be part of the standard way of doing things
  within the company.
• This provides more certainty about the
  composition of the grid, yet does not address any
  of the robustness issues that remain (PCs may
  reboot PCs may be turned off, etc.).

17
PC Distributed Computing

• PC Administration and Security
• These are generally already in place so that
  sensitive information can be distributed to the
  computational nodes without excessive concern.
  The notion of active management of PCs (for
  example, the automated push of security
  updates) is an accepted part of corporate PC
  infrastructure.

18
PC Distributed Computing

• Access to Shared Resources
• Most organizations have common data storage on
  their intranets.
• This can be as simple as a shared drive mapping
  that is established in conjunction with a network
  connection or as complex as a multi-tier,
  multi-terabyte data warehouse.
• The Desktop Grid can use this knowledge to reduce
  or eliminate redundant copies of data and to
  optimize work assignments based on the knowledge
  of which members of the Desktop Grid have access
  to which shared resources.

19
PC Distributed Computing

• The technology behind the Internet-based
  voluntary enrollment methodology for PC-based
  distributed computing combined with the needs and
  configuration of enterprise infrastructure to
  create the idea of a Desktop Grid
• as an analogous concept to a high-performance
  computing cluster based on UNIX or
  Linuxsomething that would allow commodity
  desktop PCs to be treated as a mission-critical
  corporate asset and used to do core
  compute-intensive work vital to the success of
  the company.

20
Definition

• Number of definitions of a Desktop Grid
• Depending on the assumptions made and options
  selected.
• Desktop Grid to have the following
  characteristics

21
Definition

• A defined (named) collection of machines on a
  shared network, behind a single firewall, with
  all machines running the Windows/or LINUX
  operating system.
• For simplicity, we will also assume that any
  single machine is part of oneand only
  oneDesktop Grid. This named collection may
  include dedicated machines, intermittently
  connected machines, and shared machines.

22
Definition

• A set of user-controlled policies describing the
  way in which each of these machines participates
  in the grid.
• These policies should also support automated
  addition and removal of machines without user or
  administrative intervention.

23
Definition

• A hub-and-spoke virtual network topology
  controlled by a dedicated, central server.
• In other words, the machines on the grid are
  unaware of each other except as informed by the
  central server.
• This makes the Desktop Grid computing model much
  more of a client-server architecture than a
  peer-to-peer architecture.

24
Definition

• An actively managed mechanism for distribution,
  execution, and retrieval of work to and from the
  grid under control of a central server.

25
Definition

• Assume that the intent behind the formation of
  such a Desktop Grid is to aggregate these
  resources into an easily manageable and usable
  single (virtual) resource in a fashion that
  ensures that there is little or no detectable
  degradation when these computing resources are
  used for their primary purpose while meeting
  quality of service, security, and business goals
  of the larger organization.

26
Definition

• A consistent set of terminology is needed when
  discussing Desktop Grids, their components, and
  their uses

27
Definition

• GridThis term will be used interchangeably with
  Desktop Grid for simplicity.
• Grid ServerThis is a central machine that
  controls and administers the Desktop Grid.
• Grid ClientAn individual node that is a member
  of the Desktop Grid from which spare
  computational resources will be harvested. A Grid
  Client is typically an existing desktop or laptop
  PC however, any Windows/LINUX -based PC
  connected to the corporate network can become a
  Grid Client.
• Grid Client ExecutiveThe software component of
  the grid infrastructure that resides on a PC,
  enables that PC to serve as a Grid Client, and
  manages all interaction between the Grid Client
  and the Grid Server.
• Work UnitThe packet of computation assigned to a
  Grid Client by the Grid Server. This packet
  includes a grid-enabled version of an
  application, instructions for establishing an
  environment for the application on the Grid
  Client, the input data (or a pointer to the
  location of the input data), and instructions on
  how to execute the application and produce the
  output data.

28
The Desktop Grid Value Proposition

• The collection of the existing PCs within an
  organization typically represents its single
  largest, untapped computing resource.
• Average utilization levels for PCs on the
  corporate desktop range between 5 percent and 8
  percent, yet 100 percent of the cost of
  administration and support for these PCs has
  already been factored into most corporate
  accounting schemes.
• A Desktop Grid solution creates an opportunity to
  tap into the essentially free computing
  resource represented by these underutilized PCs
• this can prove an extremely cost-effective way to
  increase computing power for most organizations.

29
The Desktop Grid Value Proposition

• A Desktop Grid is sometimes referred to as a
  virtual supercomputer. This is not far from the
  truth.
• A graph of the top 500 supercomputers in the
  world as of November 2002 (data obtained from
  www.top500.org).

30
(No Transcript)
31
The Desktop Grid Value Proposition

• Even relatively modest grids of a few thousand
  typical desktop PCs provide computing power that
  would rank among the fastest 100 supercomputers
  in the world.

32
Desktop Grid Challenges

• Entirely different characteristics than a static
  compute platform
• Intermittent AvailabilityUnlike a dedicated
  compute infrastructure, a user may choose to turn
  off or reboot his PC at any time. In addition,
  the increasing trend of using a laptop (portable)
  computer as a desktop replacement means that some
  PCs may disappear and reappear and may connect
  from multiple locations over network connections
  of varying speeds and quality.

33
Desktop Grid Challenges

• Entirely different characteristics than a static
  compute platform
• User ExpectationsThe user of the PC on the
  corporate desktop views it as a truly personal
  part of his work experience, much like a
  telephone or a stapler. It is often running many
  concurrent applications and needs to appear as if
  it is always and completely available to serve
  that employees needs.
• After a distributed computing component is
  deployed on an employees PC, that component will
  tend to be blamed for every future fault that
  occursat least until the next new component or
  application is installed.

34
Desktop Grid Challenges

• A loosely coupled network of inherently
  intermittent computing engines is an extremely
  hostile environment in which to conduct
  mission-critical computations.
• It is vital that the underlying technology of the
  Desktop Grid solution is robust in the face of
  these challenges and, where possible, turns these
  challenges into advantages.

35
Desktop Grid TechnologyKey Elements to Evaluate

• Security
• The Desktop Grid must protect the integrity of
  the distributed computation. Tampering with or
  disclosure of the application data and program
  must be prevented.
• In addition, the Desktop Grid must protect the
  integrity of the underlying computing resources.
  The Grid Client Executive must prevent
  distributed computing applications from accessing
  or modifying data on the computing resources.

36
Desktop Grid TechnologyKey Elements to Evaluate

• Multi-level Protections
• Application LevelThe distributed application
  should run on the PC in an environment that is
  completely separate from the PCs normal
  operating environment.
• This ensures the security of the PC while
  distributed application processes are running
• The Grid Clients should receive executable
  programs only from Grid Servers, which should
  always be authenticated.

37
Desktop Grid TechnologyKey Elements to Evaluate

• Multi-level Protections
• System LevelThe Grid Client Executive should
  prevent an application from using/misusing local
  or network resources. Machine configuration,
  applications, and data should be unaffected.

38
Desktop Grid TechnologyKey Elements to Evaluate

• Multi-level Protections
• Task LevelThe Grid Client Executive must encrypt
  the entire work unit to protect the integrity of
  the application, the input data, and the results.

39
Unobtrusiveness

• The Desktop Grid typically shares computing,
  storage, and network resources in the corporate
  IT environment. As a result, usage of these
  resources should be unobtrusive. The Grid Client
  should cause no degradation in PC performance,
  but should utilize every possible computing
  resource available. When the users tasks require
  any resources from the Grid Client, the Grid
  Client Executive should yield instantly and only
  resume activity as the resources again become
  available. The result is maximum utilization of
  resources with no loss of PC user productivity.

40
Openness/Ease of Application Integration

• Equally important, the Desktop Grid system must
  provide application integration in a secure
  manner, ensuring that any application misbehavior
  does not adversely affect the desktop user,
  machine configuration, or network.
• For example, this might include an inadvertent
  modal dialog box displayed by the application in
  response to an error condition.

41
Openness/Ease of Application Integration

• It is not sufficient to just allow the
  application to execute on the Grid Client
  behaviors like this must be buffered within the
  integrated application environment so that
  unobtrusive performance is achieved.
• In addition to application integration security,
  there are many issues related to overall
  application suitability for a Desktop Grid

42
Robustness

• The Desktop Grid must complete computational jobs
  with minimal failures, masking underlying
  resource and network failures.
• Given that PC resources are rarely homogeneous
  and are often intermittently available
  (especially true when the PC is a laptop), the
  grid must execute reliably with fault tolerance
  on heterogeneous resources that may be turned off
  or disconnected during execution.

43
Robustness

• Resources contributed by each PC can vary
  depending on the memory, bandwidth connection,
  type of processor, and principal use of the
  system itself.
• The Grid Server should adapt to these varying
  resources by matching and dispatching
  appropriately sized tasks to each machine.

44
Scalability

• With large numbers of PCs deployed in many
  enterprises, a grid should be capable of scaling
  to tens of thousands of PCs to take advantage of
  the increased speed and power a large grid can
  provide.
• However, grids should also scale downward,
  performing well even when the grid is limited in
  scope.

45
Central Manageability

• The Desktop Grid should provide a central network
  management capability that allows control of grid
  resources, application configuration, scheduling,
  and software version management and upgrades.
• A single system administrator should be able to
  manage all of the Grid Clients without requiring
  physical access to them.

46
Central Manageability

• Whether the grid comprises 50 or 5,000 PCs, the
  system must be manageable with no incremental
  administrative effort as new clients join the
  system. Management, queuing, and monitoring of
  the computational work should be easy and
  intuitiveconfigurable by priority, access, and
  typical run configurations.

47
Key Technology ElementsChecklists

• Security Checklist
• Disallow (or limit) access to network or local
  resources by the distributed application.
• Encrypt application and data to preserve
  confidentiality and integrity.
• Ensure that the Grid Client environment (disk
  contents, memory utilization, registry contents,
  and other settings) remains unchanged after
  running the distributed application.
• Prevent local user from interfering with the
  execution of the distributed application.
• Prevent local user from tampering with or
  deleting data associated with the distributed
  application.

48
Key Technology ElementsChecklists

• Unobtrusiveness Checklist
• Centrally manage unobtrusiveness levels that are
  changeable based on time-of-day or other factors.
• Ensure that the Grid Client Executive
  relinquishes client resources automatically.
• Ensure invisibility to local user.
• Prevent distributed application from displaying
  dialogs or action requests.
• Prevent performance degradation (and total system
  failure) due to execution of the distributed
  application.
• Require very little (ideally, zero) interaction
  with the day-to-day user of the Grid Client.

49
Key Technology ElementsChecklists

• Application Integration Checklist
• Ability to simulate a standalone environment
  within the Grid Client.
• Binary-level integration (no recompilation,
  relinking, or source code access).
• Easy integration (tools, examples, and wizards
  are provided).
• Integrated security and encryption of sensitive
  data.
• Support for any native 32-bit Windows
  application.

50
Key Technology ElementsChecklists

• Robustness Checklist
• Allocate work to appropriately configured Grid
  Clients.
• Automatically reallocate work units when Grid
  Clients are removed from grid either permanently
  or temporarily.
• Automatically reallocate work units due to other
  resource or network failures.
• Prevent aberrant applications from completely
  consuming Grid Client resources (disk, memory,
  CPU, etc.).
• Provide transparent support for all versions of
  Windows in the Grid Client population.

51
Key Technology ElementsChecklists

• Scalability Checklist
• Automatic addition, configuration, and
  registration of new Grid Clients.
• Compatible with heterogeneous resource
  population.
• Configurable over multiple geographic locations.

52
Key Technology ElementsChecklists

• Central Manageability Checklist
• Automated monitoring of all grid resources.
• Central queuing and management of work units for
  the grid.
• Central policy administration for grid access and
  utilization.
• Compatibility with existing IT management
  systems.
• Product installation and upgrade can be
  accomplished using typical enterprise software
  management environments (SMS, WinInstall, etc.).
• Remote client deployment and management.

53
Key Areas for Exploration

• Applications
• Even the most technically advanced Desktop Grid
  deployment is of little use without applications
  that can execute on it.
• To be considered as a candidate for execution on
  a Desktop Grid, you must have a LINUX/Windows
  version of the application, all supporting files
  and environmental settings needed to establish an
  execution environment for the application, and
  appropriate licensing to permit multiple,
  concurrent copies of the application to be
  executed.

54
Key Areas for Exploration

• Applications
• Certain applications are better suited for
  Desktop Grid computing than are other
  applications in fact, there is a continuum of
  Application Suitability.
• This section provides information to help you
  determine where an application resides on this
  continuum and the degree to which it is suited
  for distributed execution on a Desktop Grid
  system.

55
Key Areas for Exploration

• Application Categories
• Data ParallelThese applications process large
  input datasets in a sequential fashion with no
  application dependencies between or among the
  records of the dataset.
• Parameter SweepThese applications use an
  iterative approach to generate a multidimensional
  series of input values used to evaluate a
  particular set of output functions.
• ProbabilisticThese applications process a very
  large number of trials using randomized inputs
  (or other ab initio processes) to generate input
  values used to evaluate a particular set of
  output functions.

56
Key Areas for Exploration

• Analyzing Application Distribution Possibilities
• In all cases, it is important to realize that a
  Desktop Grid system operates by dividing (or
  distributing) the input(s) for a particular
  application the application itself is untouched
  and every work unit uses an identical copy of the
  application.
• A careful analysis of the application is required
  to understand which of its control parameters are
  hard-coded (and therefore, every work unit must
  operate using those parameters) and which are
  changeable based on input values, configuration
  files, or registry settings (and therefore, each
  work unit might operate using different values
  for those parameters).
• After considering how the application treats its
  parameters, we have two key considerations,
  regardless of application category

57
Key Areas for Exploration

• Analyzing Application Distribution Possibilities
• Understanding how to decompose the input(s) of a
  large, monolithic job into an equivalent set of
  smaller input(s) that can be processed in a
  distributed fashion
• Understanding how to recompose the output(s) from
  these smaller distributed instances of the
  application into a combined output that is
  indistinguishable from that which would have been
  produced by the single large job

58
Key Areas for Exploration

• Analyzing Application Distribution Possibilities
• We will use the term grid-enabled application
  to refer to the combination of an application
  prepared to execute on a Grid Client, a
  particular decomposition approach, and a
  particular recomposition approach.
• We can now refine our definition of Work Unit to
  be a package sent to the Grid Client containing
  the prepared application along with one member of
  the set of decomposed inputs and instructions for
  how to create one member of the set of outputs
  for recomposition.

59
Key Areas for Exploration

• ExampleData Parallel Application
• Consider an application that examines a large
  file of integers (that are stored one integer per
  line) and counts the number of items greater than
  a particular target value.
• The input to this application is the file of
  numbers and the target value the output is a
  single number.
• The large file may be split into N smaller files
  such that each line of the original file is in
  one (and only one) of the smaller files.
• The Desktop Grid system processes each of these N
  files independently using the same target value
  for the application. Calculate the sum of the
  outputs from each of the N application instances
  to re-create the output that would be generated
  by the large file.

60
Key Areas for Exploration

• ExampleParameter Sweep Application
• Consider an application that finds the maximum
  value of a function F(X,Y) over a specified range
  using an exhaustive search approach that involves
  iteration of the parameters.
• The inputs to this application are start value,
  end value, and step size for both X and Y. The
  output of this application is the largest value
  found for F along with the (X,Y) pair that
  generated that value.
• There are several ways that this application
  could be grid-enabled.
• One simple method is to launch a separate
  instance of the application for each unique value
  of X that would iterate through the entire range
  of Y while holding X constant.
• The output of each of these smaller instances
  would be an (X,Y) pair along with a value of F.
• To generate the original output, select the
  largest value of F among those returned.

61
Key Areas for Exploration

• Determining Application Suitability
• After determining how to decompose and recompose
  the application inputs and outputs for
  distributed processing, you can assess where a
  grid-enabled application falls on the continuum
  of application suitability.
• The key measurement to calculate is the Compute
  Intensity of a typical work unit this reflects
  the relative percentage of time spent moving data
  to and from the Desktop Grid Client compared to
  the time spent performing calculations on that
  data. Calculate the Compute Intensity (CI) ratio
  using this formula

62
Key Areas for Exploration

• Determining Application Suitability

63
Key Areas for Exploration

• For example, if we have a grid-enabled
  application for which a typical work unit
  executes in 15 minutes (900 seconds) on a
  hypothetical average grid client, consumes 2MB
  (2,000 KB) of input data, and produces 0.4MB (400
  KB) of output data, we can calculate the Compute
  Intensity to be
• CI (4 900) / (2000 400) 1.5

64
Key Areas for Exploration

• In general, grid-enabled applications where CI is
  greater than 1.0 are well suited for
  distributed processing using a Desktop Grid
  solution.
• However, this is not a black-and-white decision
  if you have particularly efficient data
  connectivity between your servers and your grid
  clients (for example, a 1Gb backbone with 100Mb
  desktop connectivity), then values of CI somewhat
  less than 1.0 might still yield a well-suited
  application in your environment.

65
Key Areas for Exploration

• Also, just because an application is not well
  suited does not mean that it will not workit
  means only that the overhead of moving the data
  back and forth dampens the benefit you will see
  compared to other applications.
• One of the best ways to use CI calculations is to
  help you choose between alternate ways of
  decomposing data for an application.

66
Key Areas for Exploration

• Fine-Tuning a Grid-Enabled Application
• An important factor to include in your evaluation
  of application suitability is how you plan to
  receive the benefit of using a Desktop Grid with
  that application. In general, your benefit will
  be a combination of these two factors

67
Key Areas for Exploration

• Fine-Tuning a Grid-Enabled Application
• Receiving the same answer fasterBy splitting a
  fixed amount of processing across N work units
  that can execute in parallel, you will use the
  power of a Desktop Grid to generate expected
  results more quickly.
• Receiving a better answer in the same
  timeUsing this approach, you hold your expected
  time to results constant and perform
  significantly more computations during that time.
  For example, you might explore a parameter space
  with a finer step size, dramatically increase the
  number of Monte Carlo trials, etc.

68
Key Areas for Exploration

• Fine-Tuning a Grid-Enabled Application
• Understanding which of these benefits is more
  important to you (this will vary by application)
  will help you choose work unit duration,
  input/output sizes, parameter sweep step sizes,
  etc.all of which have an effect on the CI ratio
  and make an application more or less suited for
  execution on a Desktop Grid. Weighing the various
  tradeoffs is clearly an iterative process and one
  that is more art than science.

69
Key Areas for Exploration

• Computing Environment
• There are several ways to view the introduction
  of Desktop Grids to an existing computing
  environment. In some cases, a Desktop Grid is
  considered as an alternative, lower-cost option
  when compared with acquiring new, dedicated
  computing resources.
• In other cases, a Desktop Grid is viewed as a
  complementary addition to an existing
  infrastructure in which problems with a
  Windows-based solution can be executed on the
  Desktop Grid, thereby generating capacity on the
  other compute devices for problems with only
  non-Windows solutions.
• In any case, adding a Desktop Grid to an
  organization is a substantial change to the
  overall computing environment. A partial list of
  environmental considerations includes

70
Key Areas for Exploration

• Computing Environment
• Archival and CleanupOf information on the Grid
  Server regarding completed computing tasks and
  their results.
• Backup and RecoveryIn the event of failure of a
  Grid Server (failure of a Grid Client should be
  automatically handled by the Grid Server).
• Performance Monitoring and TuningOf the Grid
  Server and the Grid Clients. Is the system
  operating in an optimal manner based on current
  usage patterns? Are employees doing their part to
  ensure maximum availability of the Grid Clients?

71
Key Areas for Exploration

• Culture
• Employee Culture
• A well-designed Desktop Grid system will be able
  to harvest all available computational cycles
  from the employees PC without any obvious
  indication of doing so, provided that the
  employee leaves the machine in a powered-on state
  (when not otherwise in use). For many
  organizations, this requires learning new habits
  and a different kind of enrollmentensuring
  that each employee with a PC on the grid
  understands the vital role his PC is playing in
  the success of the organization.

72
Key Areas for Exploration

• Culture
• Employee Culture
• Some employees may also be very concerned about
  the Grid Client Executive, perceiving it as a
  kind of spy ware or blaming it for any fault or
  failure that occurs on their PC. Such attitudes
  can be overcome with a proactive campaign of
  employee education that might include
  informational e-mails, published results (savings
  in capital expense dollars, faster time to
  results, etc.), or even a hands-on laboratory
  where the inner workings of the Grid Server,
  Client, and Executive may be seen in more detail.

73
Key Areas for Exploration

• Culture
• User Community
• Typically, consumers of large quantities of
  computing cycles are used to a very hands-on
  interaction between themselves and their
  computing devices. Current UNIX/Linux cluster
  systems generally have a fixed, physical presence
  (i.e., you can actually visit all of the CPUs) as
  well as a notion of temporary, but exclusive,
  control (i.e., I am using the cluster for the
  next two hours.). Finally, there is an
  historical prejudice against Windows-based
  devices with regard to their ability to do
  serious computation.

74
Key Areas for Exploration

• Culture
• User Community
• All of these concerns will diminish over time,
  provided the users are willing to begin using the
  Desktop Grid to solve their mission-critical
  problems. The key here is to identify one or more
  users with significant unmet computational needs
  from the current computing infrastructure
  available to them. An initial reluctance to use a
  new technology is easy to overcome based on the
  delivery of initial resultsespecially if these
  results could not have been produced in such a
  timely manner, or with such depth, using the
  previously available computing resources.

75
Additional Functionality to Consider

• We have already identified several essential
  functions provided by the Grid Server
  management/administration of all work units,
  assignment of work units to Grid Clients, and
  management/administration of all Grid Clients.
• The Grid Server can provide a number of other
  functions that will turn the Desktop Grid from a
  low-level work unit scheduling system into a
  fully automated, multifunctional, distributed
  application execution system. These include

76
Additional Functionality to Consider

• Client Group-level OperationsIn small
  (departmental) grids, administering clients on a
  one-by-one basis is relatively straightforward.
  As the size and complexity of the grid grows, it
  is more useful to administer the grid as a
  collection of virtual, overlapping groups.
• For example, one group might be all machines
  located on the second floor, another group might
  be all Windows XP machines these groups will
  have zero or more machines in common
• This notion of Client Groups must be accompanied
  by a set of rules that allow client membership to
  be determined automatically for both new Grid
  Clients and for Grid Clients that have changed
  status (for example, upgrading the Windows
  operating system on that client or adding memory
  to that client).

77
Additional Functionality to Consider

• Data CachingThe time needed to move data to and
  from the Grid Client plays an important role in
  the calculation of Computational Intensity (as
  described above).
• Advanced Desktop Grid systems will provide
  various forms of data caching in which data
  needed for a work unit can be placed in (or very
  close to) the Grid Client that will be executing
  the work unit in advance of the assignment of the
  work unit to that Client.
• This caching can either be manually controlled
  (certain data sets can be pushed to particular
  Clients and then any work unit that needs those
  data sets are assigned exclusively to those
  Clients) or automatically administered (the Grid
  Server examines its queue of work and ensures
  that any data needed for a work unit will be
  available at the Client).

78
Additional Functionality to Consider

• Job-level Scheduling and AdministrationUsers
  will want to interact with the Desktop Grid at a
  level consistent with their business problems and
  desired solutions.
• In general, the Desktop Grid system should
  support a kind of user interaction substantially
  similar to run this application using these
  inputs with this priority and put the answers
  here.
• This is substantially more abstract than the
  fundamental work unit level of the internal
  workings of the Desktop Grid.
• The Grid Server should support various levels of
  job priority along with the ability to select
  particular Clients (or groups of Clients) for a
  particular job based on characteristics of the
  job itself.

79
Additional Functionality to Consider

• Performance Tuning and AnalysisThe Desktop Grid
  system should provide all necessary data and
  reports to allow an administrator to determine
  important performance characteristics of each
  Grid Client and the grid as a whole.
• This should include optimum (theoretical)
  throughput calculations for the grid, actual
  throughput calculations for any particular job or
  set of work units, identification of any
  problematic Clients (or groups of Clients), etc.

80
Additional Functionality to Consider

• SecurityEach separately identified function
  within the Grid Server user environment should
  include user-level securitywhich users may add
  new applications, which users may submit jobs,
  which users may review job output, etc.
• The Grid Server should have its own security
  system for access to any of its components
  through direct methods (i.e., any method other
  than through the supplied user and administrative
  environments).

81
Additional Functionality to Consider

• System InterfacesThe Grid Server should support
  a variety of interfaces for its various user and
  administrative functions. At minimum, all
  functionality should be accessible through a
  browser-based interface. Other interfaces that
  might be provided include a command-line
  interface (for scripting support), a
  Windows-based API (for invoking grid
  functionality from other Windows programs), and
  an XML interface (as a general-purpose
  communication methodology). Any system interfaces
  provided in addition to the basic browser access
  must also include a security protocol.