Dept. of ECE, University of British Columbia

1
Joint Physical and Network Layer Optimization of
Communication Systems Current Challenges and
Perspectives
Dejan V. Djonin NSERC PostDoctoral Fellow Dept.
of Electrical and Computer Engineering University
of British Columbia E-mail ddjonin_at_ece.ubc.ca
www.ece.ubc.ca/ddjonin
Dept. of ECE, University of British Columbia
2
My Brief Background
(Sep 2003 - ) Postdoctoral Teaching Fellow,
University of British Columbia, Department of
Electrical and Computer Engineering (May 2000-
Jun 2003) PhD Studies, University of Victoria,
Department of Electrical and Computer
Engineering Ph.D. Thesis Title "On Some Limiting
Performance Issues of Multiuser Receivers in
Fading Channels" (1996 - 1999), Faculty of
Electrical Engineering in Belgrade, M.Sc.
studies, M.Sc. Thesis Title "Application of
Non-linear One-dimensional Maps in Generation of
Error-Correction Block Codes"
Dept. of ECE, University of British Columbia
3
UBC and Vancouver
Dept. of ECE, University of British Columbia
4
Presentation Outline

• An Overview of My Previous Professional Results
• Example Cross-layer optimization for V-BLAST
  transmission
• under delay constraints
• Problem Formulation and Introduction
• Real-Time Traffic Model Flow Control
• Channel Model Finite State Markov Model
• Mathematical Framework Stochastic Control and
  MDPs
• Solution Techniques
• Resource allocation for imperfectly known
  channel models
• Perspectives - Sensor Scheduling for Network
  Lifetime Maximization
• - Opportunistic Spectrum Access

Dept. of ECE, University of British Columbia
5
An Overview of My Previous Results (1)

• Non-linear mappings in the design of
  error-correction codes (M.Sc. Thesis)
• D.V. Djonin, D.Gacesa, "Performances of
  Error-Correction Codes Generated by Iterative
  Nonlinear Mappings", in Advances in Systems,
  Signals, Control Computers, vol. 3, Durban,
  SAR, ISBN 0-620-23136-10, pp. 114-118, 1998.
• D.V.Djonin and D.Gacesa,  "Performances of
  error-correction codes generated by non-linear
  iterative mappings", in Proc. of URSI
  International Symposium on Signals, Systems, and
  Electronics, ISSSE 98, pp. 356  360, 1998.
• D.V.Djonin and L.Manojlovich, "Application of
  deterministic chaos in generation of error
  correction block codes" , in Proc. of Second IEEE
  International Caracas Conference on Devices,
  Circuits and Systems, pp. 343-347,1998.
• D.V.Djonin, "Efficient Construction of
  Error-Correction Codes Generated by Iterative
  Non-linear Maps", in proc. of Telecommunications
  Conf. TELFOR, Belgrade Yugoslavia, 1998.
• D.V.Djonin, "On the application of the theory of
  deterministic chaos in the generation of
  error-correction codes", pp. 82-85, in Proc. of
  the ETRAN conference, Zlatibor, Yugoslavia, 1997.
• D.V.Djonin, D.Gacesa, "Performances of
  Error-Correction Codes Generated by Iterative
  Nonlinear Mappings", in Advances in Systems,
  Signals, Control Computers, vol. 3, ISBN 0-620-
  23136-10, pp. 114-118, Durban, 1998.

Dept. of ECE, University of British Columbia
6
An Overview of My Previous Results (2)

• Performance analysis and optimization of CDMA
  systems (Ph.D. Thesis)
• D.V.Djonin and V.K.Bhargava, "Asymptotic Analysis
  of the Conventional Decision Feedback Receiver in
  Fading channels'', IEEE Trans. on Wireless
  Communications, pp. 1066-1078, September 2003.
• D.V.Djonin and V.K.Bhargava, "On the Optimal
  Sequence Allocation in Flat Fading Channels'',
  IEEE Trans. on Wireless Communication, vol. 24,
  no. 5, pp. 680-689, July 2003.
• D.V.Djonin and V.K.Bhargava, "Comments on
  'Symmetric Capacity and Signal Design for
  L-out-of-K Symbol-Synchronous CDMA Gaussian
  Channels'", IEEE Trans. on Inf. Theory, pp.
  2921-2923, vol. 50, November 2004.
• D.V.Djonin and V.K.Bhargava, "Spectral Efficiency
  of the Feedback Receiver for Two Sets of
  Orthogonal Sequences" , IEEE Communication
  Letters, pp. 497-499, Nov. 2002.
• D.V.Djonin and V.K.Bhargava,  "Spectral
  Efficiency of the Feedback Receiver for Multiple
  Orthogonal Sequence Sets", in Proc. ISWC 02
  Conf., Victoria, Canada, pp.107-108, Sep. 2002.
• D.V.Djonin and V. K.Bhargava, "Asymptotic
  Analysis of the Conventional Decision Feedback
  Receiver in Flat Fading Channels'', in Proc. of
  ICC 2002, pp. 1368  1372, April-May 2002.
• D.V.Djonin and V.K.Bhargava,  "Asymptotic
  Analysis of the Optimal Spreading Sequence
  Allocation in Flat Fading Channels" , in Proc. of
  VTC 2002, pp.582-585, September 2002.
• D.V.Djonin, and V. K. Bhargava, "Low Complexity
  Receivers for Over-Saturated CDMA System'', in
  Proc. of the Globecom 2001, vol. 2, pp. 846 -850,
  Nov. 2001.

Dept. of ECE, University of British Columbia
7
An Overview of My Previous Results (3)

• Performance analysis of communications systems
  using the theory of stochastic majorization
• D.V.Djonin, P. Tarasak and V.K.Bhargava,  "On the
  Influence of the Power Delay Profile on the
  Performance of Diversity Combining Systems" ,
  Proc. of Globecom 2003 Conference, San Francisco,
  CA, vol. 3, pp. 1659 - 1663, December 2003.
• D.V.Djonin and V.K.Bhargava,  "On the Influence
  of the Power Delay Profile on the Performance of
  Diversity Combining Systems" , IEEE Transactions
  on Wireless Communications, pp. 1854-1861, vol.
  3, Sep. 2004.

Dept. of ECE, University of British Columbia
8
An Overview of My Previous Results (4)

• Results on Space-Time Code
• K. C. B. Wavegedara, D.Djonin, and V.K.Bhargava,
  "Space-Time Coded Uplink Transmission with
  Decision Feedback Sequence Estimation", in Proc.
  of Globecom 2004 Conference, Dallas, TX, vol. 6,
  pp. 3448 - 3453, Nov.-Dec. 2004.
• K. C. B. Wavegedara, D.V.Djonin, and
  V.K.Bhargava, "Space-Time Coded Uplink
  Transmission with Decision Feedback Sequence
  Estimation", in IEEE Trans on Wireless Comm., Nov
  8th, 2005. (full paper).

Dept. of ECE, University of British Columbia
9
An Overview of My Previous Results (5)

• Rate and Power Control algorithms for
  time-varying wireless channels using Markov
  Decision Processes
• A.Karmokar, D.V.Djonin and V.K.Bhargava,
  "Cross-layer Rate and Power Adaptation Strategies
  for IR-HARQ Systems over Fading Channels with
  Memory A SMDP-based Approach", submitted to
  Trans on Communications, February 21st, 2006.
• D.V.Djonin and V.Krishnamurthy, " V-BLAST Power
  and Rate Control under Delay Constraints in
  Markovian Fading Channels -Structured Policy
  Learning", submitted to IEEE Trans. on Signal
  Processing, Jan. 25, 2006.
• D.V.Djonin and V.Krishnamurthy, " V-BLAST Power
  and Rate Control under Delay Constraints in
  Markovian Fading Channels -Optimality of
  Monotonic Policies", submitted to IEEE Trans. on
  Signal Processing, Jan. 05, 2006.
• A.Karmokar, D.V.Djonin and V.K.Bhargava, "Delay
  Aware Power Adaptation for Incremental Redundancy
  Hybrid ARQ over Fading Channels with Memory", to
  be presented at the ICC 2006 Conference,
  Istanbul, Turkey.
• D.V.Djonin and V.Krishnamurthy, "Structural
  Results on the Optimal Transmission Scheduling
  Policies and Costs for Correlated Sources and
  Channels", in CDC 2005, (invited paper).
• Md.J.Hossain, D.V.Djonin and V.K.Bhargava, "Delay
  Limited Optimal and Suboptimal Power and Bit
  Loading Algorithms for OFDM Systems over
  Correlated Fading", presented at the Globecom
  2005, St. Louis, Dec. 2005.
• Md.J.Hossain, D.Djonin and V.K.Bhargava, "Power
  and Rate Adaptation for OFDM System over
  Correlated Fading Channels",  presented at the
  IST 2005 Symposium, Dresden, Germany, June 2005.
• D.V.Djonin and V.K.Bhargava, "An Upper Bound on
  the Throughput of Opportunistic Transmission in a
  Multiple-Access Fading Channel", IEEE Trans. on
  Comm., pp. 1618-1621, vol. 52, Oct. 2004.
• A.Karmokar, D.V.Djonin and V.K.Bhargava, "Optimal
  and Suboptimal Packet Scheduling over
  Time-Varying Flat Fading Channels",  to be
  published in IEEE Trans on Wireless Comm., (full
  paper), Jan., 2006.
• A.Karmokar, D.Djonin and V.K.Bhargava, "Delay
  Constrained Rate and Power Adaptation over
  Correlated Fading Channels", in Proc. of Globecom
  2004 Conference, Dallas, TX, vol. 5, pp. 2941 -
  2945, Nov.-Dec. 2004.
• D.V.Djonin, A.Karmokar and V.K.Bhargava, "Rate
  and Power Adaptation over Correlated Fading
  Channels under Different Buffer Cost
  Constraints", submitted to Trans on Vehicular
  Technology, March. 9th, 2004.
• D.V.Djonin, A. Karmokar and V.K.Bhargava, 
  "Optimal and Suboptimal Packet Scheduling over
  Time-Varying Flat Fading Channels" in Proc. of
  ICC 2004, pp. 906-910, Paris, France, June 2004.
• A.Karmokar, D.V.Djonin and V.K.Bhargava, "POMDP
  Based Coding Rate Adaptation for Hybrid ARQ
  Systems over Fading Channels with Memory",
  submitted to Trans on Wireless Communications,
  August 18th, 2004.

Dept. of ECE, University of British Columbia
10
Common Themes

• Performance Analysis of Communication Systems
• Performance Improvement of Communication Systems
  Through
• On-line and Off-line Optimization
• Main tool Stochastic Control Markov Decision
  Processes

Dept. of ECE, University of British Columbia
11
Presentation Outline

• An Overview of My Previous Professional Results
• Example Cross-layer optimization for V-BLAST
  transmission
• under delay constraints
• Problem Formulation and Introduction
• Real-Time Traffic Model Flow Control
• Channel Model Finite State Markov Model
• Mathematical Framework Stochastic Control and
  MDPs
• Solution Techniques
• Resource allocation for imperfectly known
  channel models
• Perspectives - Sensor Scheduling for Network
  Lifetime Maximization
• - Opportunistic Spectrum Access

Dept. of ECE, University of British Columbia
12
Problem Formulation and Introduction

• Modern and future wireless networks will support
  different services
• with a wide range of quality of service
  requirements such as delay, rate, BER
• Consideration of Transmission Latency is of
  crucial interest
• for some applications (real-time high quality
  audio, video transmission)
• However, time-varying nature of a wireless
  channel poses a challenging
• task to delivering a wide variety of services
• the effect is similar to congestion in wireline
  networks
• the need for transmission buffer
• transmitted signals are delayed
• Do these methods only apply to wireless
  channels?
• The solution is through adaptation of
  transmission parameters based
• on the current state and the statistical
  model of the channel and
• supported traffic

Dept. of ECE, University of British Columbia
13
Power versus Delay Tradeoff A Simple Illustration
A
B
Dept. of ECE, University of British Columbia
14
OSI Model
Data Link (Layer 2) At this layer, data packets
are encoded and decoded into bits. It furnishes
transmission protocol knowledge and management
and handles errors in the physical layer, flow
control and frame synchronization. The data link
layer is divided into two sublayers The Media
Access Control (MAC) layer and the Logical Link
Control (LLC) layer. The MAC sublayer controls
how a computer on the network gains access to the
data and permission to transmit it. The LLC layer
controls frame synchronization, flow control and
error checking. Physical (Layer 1)This layer
conveys the bit stream - electrical impulse,
light or radio signal -- through the network at
the electrical and mechanical level. It provides
the hardware and software means of sending and
receiving data on a carrier.
Dept. of ECE, University of British Columbia
15
V-BLAST transmission control model

• Let fn denote the number of packets arriving at
  the buffer in time n.
• Transmission adaptation parameters can include
  power,
• error-correction or source coding rate (flow
  control)
• At the beginning of the n-th time slot, the
  scheduler controls the packet retrievals from the
  buffer and bit-loading across carriers.

Dept. of ECE, University of British Columbia
16
Channel Model FSMC

• For example, a slowly varying flat Fading
  Rayleigh channel can be
• represented as a Finite State Markov Chain (FSMC)
  as shown in figure

• Channel can also be modeled as an Auto
  Regressive (AR) model

Dept. of ECE, University of British Columbia
17
Presentation Outline

• An Overview of My Previous Professional Results
• Example Cross-layer optimization for V-BLAST
  transmission
• under delay constraints
• Problem Formulation and Introduction
• Real-Time Traffic Model Flow Control
• Channel Model Finite State Markov Model
• Mathematical Framework Stochastic Control and
  MDPs
• Solution Techniques
• Resource allocation for imperfectly known
  channel models
• Perspectives - Sensor Scheduling for Network
  Lifetime Maximization
• - Opportunistic Spectrum Access

Dept. of ECE, University of British Columbia
18
Markov Decision Processes (MDP)
Markov Chain Example
p(S2S1)
p(S2S2)
S1
S2
p(S1S1)
p(S1S2)
Markov Decision Processes Example for state S1
Action U1, c(S1,U1)
p(S2S1,U1)
Action U2,c(S1,U2)
p(S2S1,U2)
p(S1S1,U1)
S1
S2
p(S1S1,U2)
Dept. of ECE, University of British Columbia
19
Constrained MDPs

• What happens if in addition to the immediate
  costs, c(s,u), there is an another cost d(s,u)
  that corresponds to a constraint? I.e.
  optimization problem is
• The answer can be found in the theory of
  Constrained Markov Decision Processes (CMDP).
  CMDP can be expressed as equivalent
  unconstrained MDP using Lagrangian Approach
• Note that policies do not have to be
  deterministic in CMDPs. In general optimal
  policies for CMDPs are randomized.

Dept. of ECE, University of British Columbia
20
V-BLAST transmission control model

• Let fn denote the number of packets arriving at
  the buffer in time n.
• Transmission adaptation parameters can include
  power,
• error-correction or source coding rate (flow
  control)
• At the beginning of the n-th time slot, the
  scheduler controls the packet retrievals from the
  buffer and bit-loading across carriers.

Dept. of ECE, University of British Columbia
21
Presentation Outline

• An Overview of My Previous Professional Results
• Example Cross-layer optimization for V-BLAST
  transmission
• under delay constraints
• Problem Formulation and Introduction
• Real-Time Traffic Model Flow Control
• Channel Model Finite State Markov Model
• Mathematical Framework Stochastic Control and
  MDPs
• Solution Techniques
• Resource allocation for imperfectly known
  channel models
• Perspectives - Sensor Scheduling for Network
  Lifetime Maximization
• - Opportunistic Spectrum Access

Dept. of ECE, University of British Columbia
22
Sample Results (1)

• As fading rate ?, the rate of decrease of
  average power ?.
• As the number of antennas ?, average power ?

Dept. of ECE, University of British Columbia
23
Sample Results (2)
Dept. of ECE, University of British Columbia
24
Structural Results
Extracted from the paper MIMO Power and Rate
Control under Delay Constraints in Markovian
Fading Channels Optimality of Monotonic
Policies, Dejan V. Djonin, Vikram Krishnamurthy,
submitted to Trans. on Signal Processing, Jan
2006, revised May 2006. also to be presented at
the ISIT Conference, Seattle 2006.
Dept. of ECE, University of British Columbia
25
Resource allocation for imperfectly known channel
models (1)

• This a challenging problem as the policy has to
  be learned on-line as the actions are being
  applied and observations on the incurred cost are
  collected.
• The appropriate framework for the solution of
  this problem is to consider Q-learning, which is
  a version of stochastic approximation algorithm.
• For details on Q-algorithm and related topics
  have a look at
• D. Bertsekas and J.Tsitsiklis, Neuro-Dynamic
  Programming

Dept. of ECE, University of British Columbia
26
Resource allocation for imperfectly known channel
models (2)
Extracted from the Paper Dejan Djonin, Vikram
Krishnamurthy, V-BLAST Power and Rate Control
under Delay Constraints in Markovian Fading
Channels- Structured Policy Learning Algorithm,
submitted to Trans on Signal Processing, Jan 2006.
Dept. of ECE, University of British Columbia
27
Resource allocation for imperfectly known channel
models (3)

• Advantages of Learning based algorithms for
  Optimal Control
• It can be proved that Q-learning algorithm
  converges to the optimal solution with
  probability 1 (both structured and non-structured
  Q-learning)
• These algorithms are suitable for unknown
  channel environments whose statistics changes
  slowly over time
• It is possible to incorporate more complicated
  delay costs in the model average delay cost,
  maximum delay guarantees, delay profile shaping

Dept. of ECE, University of British Columbia
28
Presentation Outline

• An Overview of My Previous Professional Results
• Example Cross-layer optimization for V-BLAST
  transmission
• under delay constraints
• Problem Formulation and Introduction
• Real-Time Traffic Model Flow Control
• Channel Model Finite State Markov Model
• Mathematical Framework Stochastic Control and
  MDPs
• Solution Techniques
• Resource allocation for imperfectly known
  channel models
• Perspectives - Sensor Scheduling for Network
  Lifetime Maximization
• - Opportunistic Spectrum Access

Dept. of ECE, University of British Columbia
29
Sensor Scheduling for Network Lifetime
Maximization
h1
hN
h2
eN
e1
e2
Sensor 1
Sensor N
Sensor 2
Collaborators Qing Zhao, Yunxia Chen (UC Davis),
V.Krishnamurthy(UBC)
Dept. of ECE, University of British Columbia
30
Sensor Scheduling for Network Lifetime
Maximization

• The problem is how to design an optimal sensor
  scheduling policy to
• maximize the lifetime of a network as a whole
• The sensor network is considered to be
  functioning while a predefined portion of sensors
  have enough energy to transmit
• Transmission energy is dependent on the channel
  conditions Wi f(hi)
• Two approaches to model and solve the problem
• centralized scheduling, global state MDP
• decentralized scheduling, multi-armed bandit
  formulation
• Some results on this topic are given in
• 1) Y. Chen, Q. Zhao, V. Krishnamurthy and
  D.V.Djonin, "Transmission Scheduling for
  Optimizing Sensor Network Lifetime A Stochastic
  Shortest Path Approach", submitted to IEEE Trans.
  on Signal Processing, Jan. 2006, revised May
  2006.
• 2) Y. Chen, Q. Zhao, V. Krishnamurthy and
  D.V.Djonin, "Transmission Scheduling for Sensor
  Network Lifetime Maximization A Shortest Path
  Bandit Formulation", presented at the ICASSP 2006
  Conference, Toulouse, France, May 2006.

Dept. of ECE, University of British Columbia
31
Sensor Scheduling for Network Lifetime
Maximization Open Problems

• Further simplification of the computation of the
  optimal sensor scheduling policy for centralized
  scheduling
• Incorporation of the content based scheduling
  (the information sent by different schedulers can
  be prioritized)
• Adaptive Source Coding Control prior to
  transmission
• Multiple Access transmission resolution

Dept. of ECE, University of British Columbia
32
Opportunistic Spectrum Access
Channel 1
Channel 2
Channel 3
Channel 4
Channel 5
Channel 6
Channel 7
B1
B2
B3
B4
B5
B6
B7
p1
p2
p4
p5
p6
p7
p3
Scheduler f(p1,, p7)
pi Prob channel i is available
Bi Bandwidth of the Channel i
Collaborator Qing Zhao (UC Davis)
Dept. of ECE, University of British Columbia
33
Opportunistic Spectrum Access Open Problems

• Design of a computationally efficient Spectrum
  Access control policy
• Exploration of the decentralized formulation of
  the problem a restless multi-armed bandit
  formulation
• Protocol design for coordination between primary
  and secondary users

Dept. of ECE, University of British Columbia
34
Thank You for Your Attention !
Dept. of ECE, University of British Columbia