Parallel Computation

1
Parallel Computation

Doing more than one thing at a time
2
Why Parallel Computation

• nowadays most computers contain more than a
  single processing element
• multi-core processors
• multi-processor architectures
• serial program is executed on a single processor
• programmer may take advantage of multiple
  processing elements by writing a parallel program
• task needs to be parallelizable several
  portions of code can be computed concurrently
• program needs to be coded to explicitly state
  which code to run concurrently

3
Concurrent Vector

• note that some operations (outputting to cout)
  and data structures (vectors) are not
  concurrency-safe concurrently accessing them
  may lead to unpredictable results
• concurrent vector class is concurrency safe
• to use need to include concurrent_vector.h header
• to declare concurrent_vector ltbaseTypegt name
• example
• concurrent_vector ltintgt v

4
Parallel Patterns Library (PPL)

• implemented by MSVS 2010
• not part of any of C standards
• to use need to
• include ppl.h standard header
• use Concurrency namespace (note the capital
  letter)
• supported construct parallel_for
• syntax parallel_for (startValue, endValue,
  function)
• for each index value between startValue and
  endValue-1 concurrently executes function, value
  is passed as a single parameter to the function
• watch out debugger treats parallel_for as a
  single statement
• good idea to debug with regular for-loop before
  converting to parallel_for

5
Lambda Expressions

• part of C11 a new C standard.
• Implemented by MSVS 2010
• not implemented by gdb yet. Do not try on Unix
  machines.
• anonymous function
• syntax capture(parameters)body
• capture passing discipline and (optionally)
  name of variables taken from outside of scope of
  the lambda expressions
• no variables defined. Attempting to use any
  external variables in the lambda is an error
• x, y x is captured by value, y is captured by
  reference
• any external variable is implicitly captured
  by reference
• any external variable is implicitly captured
  by value
• convenient construct to be used with parallel_for

6
Concurrent Vector Fill Example

• concurrent_vectorltintgt a int size10
• parallel_for(0, 10, (int value)
• a.push_back(value)
• )
• values may be added to the vector out of order
• no performance gain push back is a serial
  operation
• can you write the code that concurrently doubles
  every element of the vector?

7
Prefix Sum

• a common concurrent algorithm for concurrently
  examining all elements in an array. For example,
  computing a sum.
• idea can add pairs of elements concurrently,
  then pairs of pairs,

5
12
1
4
8
6
14
2
7
3
9
17
5
14
16
10
9
22
30
19
52
19
71
8
Sequential Prefix Sum

• for (int gap1 gap lt a.size() gap 2)
• for (int i0 i lt a.size() i)
• if ( i (gap2) 0 igap lt a.size())
• ai aigap

5
12
1
4
8
6
14
2
7
3
9
17
5
14
16
10
9
i
gap
22
30
19
i element collecting the sum gap distance
between added elements
52
19
71
9
Parallel Prefix Sum

• for (int gap1 gap lt a.size() gap 2)
• parallel_for (0, int(a.size()), (int i)
• if ( i (gap2) 0 igap lt a.size())
• ai aigap
• )

5
12
1
4
8
6
14
2
7
3
9

• how many times is parallel_for invoked?
• how do you compute the average value of the
  vector?
• the minimum? the maximum?

17
5
14
16
10
9
22
30
19
52
19
71
10
Questions on Parallel Computation

• Why is parallel computation needed?
• What is the difference between parallel and
  serial program? What is concurrency?
• What is concurrent_vector? How is it different
  from regular vector?
• What is parallel patterns library?
• What is lambda expressions? What is capture? What
  is the difference between and ?
• What is parallel_for? what are startValue and
  endValue, function parameters in parallel_for?
• What is prefix sum algorithm? Why is it efficient
  to run concurrently? What values can be computed
  by this algorithm?