4  Chapter 3: Random Processes

4.1 Module 3: Random Processes

4.1.1 Topic Summary

This module delves into random processes over time, focusing on Wide-Sense Stationary (WSS) processes where the mean is constant and autocorrelation depends only on time differences \(\tau\), exemplified by sinusoidal processes with random amplitudes/phases. Autocorrelation \(R_{XX}(\tau)\) measures similarity at different lags, with \(R_{XX}(0)\) giving mean square value. Power Spectral Density (PSD) is the Fourier transform of autocorrelation, revealing frequency components. Poisson processes model random arrivals with exponential inter-arrival times (mean \(1/\lambda\)), used for call arrivals, message counts, or waiting intervals, with probabilities calculated using the Poisson PMF for counts or exponential CDF for times (Papoulis and Pillai 2002).

Tips for Beginners

  • WSS: Constant mean, autocorrelation R(τ) depends on τ only. Think of it as “time-invariant” statistics.

  • Autocorrelation: \(R_{XX}(\tau) = E[X(t) X(t+\tau)]\). For WSS, it’s symmetric \(R_{XX}(\tau) = R_{XX}(-\tau)\).

  • PSD: Fourier transform of \(R_{XX}(\tau)\). High PSD at frequencies means strong components there.

  • Poisson Process: Arrivals independent, exponential inter-arrivals. For counts in interval, use Poisson; for times, exponential.

  • Common Mistakes: Don’t confuse Poisson (discrete counts) with Exponential (continuous times). For PSD, remember it’s complex but often real for real processes.

  • Practice: Visualize processes as time series. Use Poisson for “rare events” like calls, Exponential for “waiting”.

4.1.2 Solved Problems

  1. Defining WSS: Define a Wide-Sense Stationary (WSS) process.

    • Solution: A random process \(X(t)\) is WSS if it satisfies two conditions: (1) Its expected value \(E[X(t)]\) is a constant, and (2) its autocorrelation function \(R_{XX}(t_1, t_2)\) depends only on the time difference \(\tau = t_2 - t_1\).

  2. Mean Square Value: If the autocorrelation function of a WSS process is \(R_{XX}(\tau) = 25 + \frac{4}{1+6\tau^2}\), find the mean square value of the process.

    • Solution: The relationship states that Mean Square Value \(E[X^2(t)] = R_{XX}(0)\). Substituting \(\tau=0\) yields \(R_{XX}(0) = 25 + \frac{4}{1+0} = 29\).

  3. Poisson Process Arrivals: Calls arrive at a station according to a Poisson process with an average of 12 per minute. Find the probability that more than 3 calls arrive in a 20-second interval.

    • Solution: Convert the rate to the 20-second interval: \(\lambda = (12/60) \times 20 = 4\). The probability is \(P(X > 3) = 1 - P(X \le 3) = 1 - e^{-4}(4^0/0! + 4^1/1! + 4^2/2! + 4^3/3!) \approx 0.5665\).

  4. PSD to Autocorrelation: Find the autocorrelation function \(R_{XX}(\tau)\) if the power spectral density is \(S_{XX}(\omega) = \frac{8}{16+\omega^2}\).

    • Solution: Applying the Inverse Fourier Transform formula: \(R_{XX}(\tau) = \frac{1}{2\pi} \int S_{XX}(\omega) e^{j\omega \tau} d\omega\). Solving this integral gives \(R_{XX}(\tau) = e^{-4|\tau|}\).

  5. Poisson Process Wait Times: If customers arrive at a counter at a rate of 10 per hour, find the probability that the interval between two successive arrivals is more than 15 minutes.

    • Solution: Inter-arrival times in a Poisson process follow an exponential distribution. Here, \(T \sim Exp(10)\). 15 minutes is \(0.25\) hours. \(P(T > 0.25) = e^{-10(0.25)} = e^{-2.5} \approx 0.082\).

  6. WSS Process Example: Let \(X(t) = A \cos(\omega t) + B \sin(\omega t)\), where \(A\) and \(B\) are independent random variables with mean 0 and variance \(\sigma^2\). Show that \(X(t)\) is WSS.

    • Solution: \(E[X(t)] = 0\). The autocorrelation function simplifies to \(R_{XX}(\tau) = \sigma^2 \cos(\omega \tau)\), confirming it is WSS.

  7. Power Spectral Density Definition: Define the Power Spectral Density of a Wide Sense Stationary (WSS) process.

    • Solution: It is the Fourier Transform of the autocorrelation function \(R_{XX}(\tau)\).

  8. Poisson Process Messages: Messages arrive at a computer center according to a Poisson process with a mean rate of 5 per hour. Find the probability that exactly 2 messages arrive in 30 minutes.

    • Solution: For 30 mins, \(\lambda = 2.5\). \(P(X=2) = \frac{e^{-2.5}(2.5)^2}{2!} \approx 0.2565\).

4.1.3 Practice Problems

  • Show that the random process \(X(t) = Y \cos(\omega t + \theta)\) is WSS if Y is a WSS process and \(\theta\) is uniformly distributed in \((0, 2\pi)\).

  • Find the power spectral density of a random process \(X(t) = A \sin(\omega_0 t + \theta)\) where \(\theta\) is uniform in \((0, 2\pi)\).

  • For a Poisson process with parameter \(\lambda\), show that the time between successive arrivals follows an exponential distribution with mean \(1/\lambda\).

  • A WSS process has an autocorrelation function \(R_{XX}(\tau) = 16 \cos(10\tau)\). Find the mean square value of the process.

Exam Preparation Tips

  • WSS conditions: E[X(t)] constant, R(t1,t2) = R(τ).

  • For Poisson, remember to adjust λ for the time interval. Use PMF for counts, CDF for times.

  • For PSD, practice Fourier transforms of common functions to get R(τ).

  • Poisson: P(N(t)=k) = e^{-λt} (λt)^k / k!, Inter-arrival ~ Exp(λ).

  • PSD: S(ω) = ∫ R(τ) e^{-jωτ} dτ.

  • Practice: Identify if process is WSS by checking mean and R.

  • Common pitfall: Exponential for times, Poisson for counts in Poisson process.