Title: High-Performance Computing, Computational Science, and NeuroInformatics Research
1High-Performance Computing, Computational
Science, and NeuroInformatics Research
- Allen D. Malony
- Department of Computer and Information Science
- NeuroInformatics Center (NIC)
- Computational Science Institute
- University of Oregon
2Outline
- High-performance computing research
- Interactions and funding
- Project areas
- TAU parallel performance system
- Computational science at UO
- Projects
- Computational Science Institute
- Neuroinformatics research
- NeuroInformatics Center (NIC)
- ICONIC Grid
3High-Performance Computing Research
- Strong associations with DOE national
laboratories - Los Alamos National Lab
- Lawrence Livermore National Lab
- Sandia National Lab (Livermore)
- Argonne National Lab
- National Energy Research Supercomputing Center
- DOE funding
- Office of Science, Advance Scientific Computing
- ASCI/NNSA
- NSF funding
- Academic Research Infrastructure
- Major Research Instrumentation
4Project Areas
- Parallel performance evaluation and tools
- Parallel language systems
- Tools for parallel system and software
interaction - Source code analysis
- Parallel component software
- Computational services
- Grid computing
- Parallel modeling and simulation
- Scientific problem solving environments
5TAU Parallel Performance System
- Allen D. Malony Sameer S. Shende
- Department of Computer and Information Science
- Computational Science Institute
- University of Oregon
6Parallel Performance Research
- Tools for performance problem solving
- Empirical-based performance optimization process
PerformanceTuning
hypotheses
Performance Diagnosis
properties
Performance Experimentation
characterization
Performance Observation
7Complexity Challenges for Performance Tools
- Computing system environment complexity
- Observation integration and optimization
- Access, accuracy, and granularity constraints
- Diverse/specialized observation
capabilities/technology - Restricted modes limit performance problem
solving - Sophisticated software development environments
- Programming paradigms and performance models
- Performance data mapping to software abstractions
- Uniformity of performance abstraction across
platforms - Rich observation capabilities and flexible
configuration - Common performance problem solving methods
8General Problems
- How do we create robust and ubiquitous
performance technology for the analysis and
tuning of parallel and distributed software and
systems in the presence of (evolving) complexity
challenges? -
- How do we apply performance technology
effectively for the variety and diversity of
performance problems that arise in the context of
complex parallel and distributed computer systems?
?
9TAU Performance System
- Tuning and Analysis Utilities
- Performance system framework for scalable
parallel and distributed high-performance
computing - Targets a general complex system computation
model - nodes / contexts / threads
- Multi-level system / software / parallelism
- Measurement and analysis abstraction
- Integrated toolkit for performance
instrumentation, measurement, analysis, and
visualization - Portable performance profiling and tracing
facility - Open software approach with technology
integration - University of Oregon , Forschungszentrum Jülich,
LANL
10TAU Performance System Architecture
11TAU Performance System Status
- Computing platforms
- IBM SP / Power4, SGI Origin 2K/3K, ASCI Red, Cray
T3E / SV-1 / X-1, HP (Compaq) SC (Tru64), HP
Superdome (HP-UX), Sun, Hitachi SR8000, NEX
SX-5/6, Linux clusters (IA-32/64, Alpha, PPC,
PA-RISC, Power, Opteron), Apple (G4/5, OS X),
Windows - Programming languages
- C, C, Fortran 77/90/95, HPF, Java, OpenMP,
Python - Communication libraries
- MPI, PVM, Nexus, shmem, LAMPI, MPIJava
- Thread libraries
- pthreads, SGI sproc, Java,Windows, OpenMP
12TAU Performance System Status (continued)
- Compilers
- Intel KAI (KCC, KAP/Pro), PGI, GNU, Fujitsu, Sun,
Microsoft, SGI, Cray, IBM (xlc, xlf), Compaq,
Hitachi, NEC, Intel - Application libraries (selected)
- Blitz, A/P, PETSc, SAMRAI, Overture, PAWS
- Application frameworks (selected)
- POOMA, MC, ECMF, Uintah, VTF, UPS, GrACE
- Performance technology integrated with TAU
- PAPI, PCL, DyninstAPI, mpiP, MUSE/Magnet
- TAU full distribution (Version 2.x, web download)
- TAU performance system toolkit and users guide
- Automatic software installation and examples
13Computational Science
- Integration of computer sciencein traditional
sciencedisciplines - Third model ofscientificresearch
- Application ofhigh-performancecomputation,
algorithmsand networking - Parallel computing
- Grid computing
14Computational Science Projects at UO
- Geological science
- Model coupling for hydrology
- Bioinformatics
- Zebrafish Information Network (ZFIN)
- Evolution of gene families
- Oregon Bioinformatics Tool
- Neuroinformatics
- Electronic notebooks
- Domain-specific problem solving environments
- Dinosaur skeleton and motion modeling
- Computational Science Institute
15Computational Science ? Cognitive Neuroscience
- Computational methods applied to scientific
research - High-performance simulation of complex phenomena
- Large-scale data analysis and visualization
- Understand functional activity of the human
cortex - Multiple cognitive, clinical, and medical domains
- Multiple experimental paradigms and methods
- Need for coupled/integrated modeling and analysis
- Multi-modal (electromagnetic, MR, optical)
- Physical brain models and theoretical cognitive
models - Need for robust tools computational informatic
16Brain Dynamics Analysis Problem
- Identify functional components
- Different cognitive neuroscience research
contexts - Clinical and medical applications
- Interpret with respect to physical and cognitive
models - Requirements spatial (structure), temporal
(activity) - Imaging techniques for analyzing brain dynamics
- Blood flow neuroimaging (PET, fMRI)
- good spatial resolution ? functional brain
mapping - temporal limitations to tracking of dynamic
activities - Electromagnetic measures (EEG/ERP, MEG)
- msec temporal resolution to distinguish
components - spatial resolution sub-optimal (source
localization)
17Integrated Electromagnetic Brain Analysis
good spatial poor temporal
Cortical Activity Knowledge Base
Head Analysis
Source Analysis
Structural / Functional MRI/PET
spatial pattern recognition
temporal dynamics
Cortical Activity Model
Experiment subject
IndividualBrain Analysis
Constraint Analysis
Component Response Model
neural constraints
Dense Array EEG / MEG
temporal pattern recognition
Signal Analysis
Response Analysis
Component Response Knowledge Base
poor spatial good temporal
neuroimaging integration
18Experimental Methodology and Tool Integration
16x256bits permillisec (30MB/m)
CT / MRI
segmentedtissues
EEG
NetStation
BrainVoyager
processed EEG
mesh generation
source localization constrained to cortical
surface
Interpolator 3D
EMSE
BESA
19NeuroInformatics Center (NIC)
- Application of computational science methods to
cognitive and clinical neuroscience problems - Understand functional activity of the brain
- Help to diagnosis brain-related disorders
- Utilize high-performance computing and simulation
- Support large-scale data analysis and
visualization - Advance techniques for integrated neuroimaging
- Coupled modeling (EEG/ERP and MR analysis)
- Advanced statistical factor analysis
- FDM/FEM brain models (EEG, CT, MRI)
- Source localization
- Problem-solving environment for brain analysis
20NIC Organization
- Director, Allen D. Malony
- Associate Professor, Computer and Information
Science - Associate Director, Don M. Tucker
- Professor, Psychology CEO, EGI
- Computational Scientist, Kevin Glass
- Ph.D., Computer Science B.S., Physics
- Computational Physicist, Sergei Turovets
- Ph.D., Computer Science B.S., Physics
- Computer Scientist, Sameer S. Shende
- Ph.D., Computer Science parallel computing
specialist - Mathematician, Bob Frank
- M.S., Mathematics
21Funding Support
- BBMI federal appropriation
- DoD Telemedicine Advanced Technology Research
Command (TATRC) - Initial budget of approximately 750K
- Oct. 1, 2002 through March 31, 2004
- NSF Major Research Instrumentation
- ICONIC Grid, awarded
- New proposal opportunities
- NIH Human Brain Project Neuroinformatics
- NSF ITR
22NIC Approaches
- Optimize spatial resolution
- MRI structural information
- Measurement of skull conductivity
- Convergence / co-recording with MEG and fMRI
- Optimize temporal resolution
- Use EEG/MEG time course for fMRI signal
extraction - Decomposition of component analysis (ICA, PCA)
- Single-trial analysis
- Computational brain models
- Boundary and finite element brain models
- Brain information databases and atlases
23EEG/ERP Methodology
- Electroencephalogram (EEG)
- Event-Related Potential (ERP)
- Stimulus-locked measures of brain dynamics
- Generated from subject- and trial-based analysis
- Raw EEG datasets processed and analyzed
- Segmentation to time series waveforms
- Blink removal and other cleaning
- ERP analysis
- Averaging for increasing signal to noise
- Characterization with respect to trial conditions
- Results visualization
- Source localization
24EGI Geodesics Sensor Net
- Electrical Geodesics Inc.
- Dense-array sensor technology
- 64/128/256 channels
- 256-channel geodesics sensor net
- AgCl plastic electrodes
- Carbon fiber leads
- Future optical sensors
- EGI LANL
25EEG/ERP Experiment Management System
- Support EEG-based cognitive neuroscience research
- Based on experiment model
- Experiment type
- Subjects measured for trial types
- Management of experiment data
- Raw and processed datasets and derived statistics
- Per experiment/subject/trial database
- Secure protection and storage with selective
access - Analysis tools and workflows
- Generation of results (across experimental
variables) - Analysis processes with multi-tool workflows
26EEG/ERP Experiment Analysis Environment
processed datasets / derived results
raw
analysis workflow
virtual services
storage resources
compute resources
27Source Localization
- Mapping of scalp potentials to cortical
generators - Single time sample and time series
- Requirements
- Accurate head model and physics
- High-resolution 3D structural geometry
- Precise tissue identification and segmentation
- Correct tissue conductivity assessment
- Computational head model formulation
- Finite element model (FEM)
- Finite difference model (FDM)
- Forward problem calculation
- Dipole search strategy
28Advanced Image Segmentation
- Native MR gives high gray-to-white matter
contrast - Edge detection finds region boundaries
- Segments formed by edge merger
- Color depicts tissue type
- Investigate more advance level set methods and
hybrid methods
29Building Finite Element Brain Models
- MRI segmentation of brain tissues
- Conductivity model
- Measure head tissue conductivity
- Electrical impedance tomography
- small currents are injectedbetween electrode
pair - resulting potential measuredat remaining
electrodes - Finite element forward solution
- Source inverse modeling
- Explicit and implicit methods
- Bayesian methodology
scalp
CSF
skull
cortex
30Conductivity Modeling
Governing Equations ICS/BCS
Continuous Solutions
Finite-DifferenceFinite-ElementBoundary-Element
Finite-VolumeSpectral
Discretization
System of Algebraic Equations
Discrete Nodal Values
TridiagonalADISORGauss-SeidelGaussian
elimination
Equation (Matrix) Solver
? (x,y,z,t)J (x,y,z,t)B (x,y,z,t)
Approximate Solution
31Source Localization Analysis Environment
raw
virtual services
storage resources
compute resources
32NIC Computational Cluster (Neuronic Cluster)
- Dell computational cluster
- 16 dual-processor nodes
- 2.8 MHz Pentium Xeon
- 4 Gbyte memory
- 36 Gbyte disk
- Dual Gigabit ethernet adaptors
- 2U form factor
- Master node (same specs)
- 2 Gigabit ethernet switches
- Brain modeling
- Component analysis
33NIC Relationships
OHSU/ OGI
Utah
LANL
Argonne
UCSD
Internet2
NCSA
Sandia
USC
Academic
Labs / Centers
Intel
IBM
UO Departments
EGI
NIC
Psychology
BDL
BEL
Industry
UO Centers/Institutes
CIS
Physics
CSI
CDSI
BBMI
CNI
NSI
34NSF MRI Proposal
- Major Research Instrumentation (MRI)
- Acquisition of the Oregon ICONIC Grid for
Integrated COgnitive Neuroscience Informatics and
Computation - PIs
- Computer Science Malony, Conery
- Psychology Tucker, Posner, Nunnally
- Senior personnel
- Computer Science Douglas, Cuny
- Psychology Neville, Awh, White
- Approximately 1.2M over three years
35ICONIC Grid
graphics workstations
interactive, immersive viz
other campus clusters
Internet 2
Gbit Campus Backbone
CNI
NIC
NIC
CIS
CIS
4x8
16
16
2x8
2x16
SMP Server IBM p655
Graphics SMP SGI MARS
Shared Memory IBM p690
Distributed Memory IBM JS20
Distributed Memory Dell Pentium Xeon
5 Terabytes
SAN Storage System
36Cognitive Neuroscience and ICONIC Grid
- Common questions to be explored
- Identifying brain networks
- Critical periods during normal development
- Network involvement in psychopathologies
- Training interventions in network development
- Research areas
- Development of attentional networks
- Brain plasticity in normal development and
deprived - Attention and emotion regulation
- Spatial working memory and selective attention
- Attention and psychopathology
37Computer Science and ICONIC Grid
- Scheduling and resource management
- Assign hardware resources to computation tasks
- Scheduling of workloads for
- PSEs for computational science
- Provide scientists an entrée to the computational
and data management power of the infrastructure
without requiring specialized knowledge of
parallel execution - Marine seismic tomograph, molecular evolution
- Interactive / immersive three-dimensional
visualization - Explore multi-sensory visualization
- Merge 3D graphics with force-feedback haptics
- Parallel performance evaluation