Audio Processing is likely one of the most vital utility domains of digital sign processing (DSP) and machine studying. Modeling acoustic environments is a necessary step in creating digital audio processing methods reminiscent of: speech recognition, speech enhancement, acoustic echo cancellation, and so on.
Acoustic environments are crammed with background noise that may have a number of sources. For instance, when sitting in a espresso store, strolling down the road, or driving your automobile, you hear sounds that may be thought-about as interference or background noise. Such interferences don’t essentially observe the identical statistical mannequin, and therefore, a combination of fashions may be helpful in modeling them.
These statistical fashions will also be helpful in classifying acoustic environments into completely different classes, e.g., a quiet auditorium (class 1), or a barely noisier room with closed home windows (class 2), and a 3rd choice with home windows open (class 3). In every case, the extent of background noise may be modeled utilizing a combination of noise sources, every taking place with a unique chance and with a unique acoustic degree.
One other utility of such fashions is within the simulation of acoustic noise in numerous environments based mostly on which DSP and machine studying options may be designed to unravel particular acoustic issues in sensible audio methods reminiscent of interference cancellation, echo cancellation, speech recognition, speech enhancement, and so on.
A easy statistical mannequin that may be helpful in such situations is the Gaussian Mixture Model (GMM) during which every of the completely different noise sources is assumed to observe a particular Gaussian distribution with a sure variance. All of the distributions may be assumed to have zero imply whereas nonetheless being sufficiently correct for this utility, as additionally proven on this article.
Every of the GMM distributions has its personal chance of contributing to the background noise. For instance, there might be a constant background noise that happens more often than not, whereas different sources may be intermittent, such because the noise coming via home windows, and so on. All this needs to be thought-about in our statistical mannequin.
An instance of simulated GMM knowledge over time (normalized to the sampling time) is proven within the determine under during which there are two Gaussian noise sources, each of zero imply however with two completely different variances. On this instance, the decrease variance sign happens extra typically with 90% chance therefore the intermittent spikes within the generated knowledge representing the sign with larger variance.
In different situations and relying on the applying, it might be the opposite approach round during which the excessive variance noise sign happens extra typically (as might be proven in a later instance on this article). Python code used to generate and analyze GMM knowledge may even be proven later on this article.
Turning to a extra formal modelling language, let’s assume that the background noise sign that’s collected (utilizing a high-quality microphone for instance) is modeled as realizations of impartial and identically distributed (iid) random variables that observe a GMM as proven under.
The modeling drawback thus boils all the way down to estimating the mannequin parameters (i.e., p1, σ²1, and σ²2) utilizing the noticed knowledge (iid). On this article, we might be utilizing the method of moments (MoM) estimator for such goal.
To simplify issues additional, we are able to assume that the noise variances (σ²1 and σ²2) are identified and that solely the blending parameter (p1) is to be estimated. The MoM estimator can be utilized to estimate a couple of parameter (i.e., p1, σ²1, and σ²2) as proven in Chapter 9 of the e book: “Statistical Sign Processing: Estimation Principle”, by Steven Kay. Nevertheless, on this instance, we’ll assume that solely p1 is unknown and to be estimated.
Since each gaussians within the GMM are zero imply, we’ll begin with the second second and attempt to get hold of the unknown parameter p1 as a operate of the second second as follows.
Be aware that one other easy methodology to acquire the moments of a random variable (e.g., second second or larger) is by utilizing the second producing operate (MGF). A superb textbook in chance principle that covers such matters, and extra is: “Introduction to Chance for Information Science”, by Stanley H. Chan.
Earlier than continuing any additional, we want to quantify this estimator when it comes to the elemental properties of estimators reminiscent of bias, variance, consistency, and so on. We’ll confirm this later numerically with a Python instance.
Beginning with the estimator bias, we are able to present that the above estimator of p1 is certainly unbiased as follows.
We will then proceed to derive the variance of our estimator as follows.
It’s also clear from the above evaluation that the estimator is constant since it’s unbiased and likewise its variance decreases when the pattern dimension (N) will increase. We may even use the above method of the p1 estimator variance in our Python numerical instance (proven intimately later on this article) when evaluating principle with sensible numerical outcomes.
Now let’s introduce some Python code and do some enjoyable stuff!
First, we generate our knowledge that follows a GMM with zero means and customary deviations equal to 2 and 10, respectively, as proven within the code under. On this instance, the blending parameter p1 = 0.2, and the pattern dimension of the information equals 1000.
# Import the Python libraries that we'll want on this GMM instance
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
# GMM knowledge era
mu = 0 # each gaussians in GMM are zero imply
sigma_1 = 2 # std dev of the primary gaussian
sigma_2 = 10 # std dev of the second gaussian
norm_params = np.array([[mu, sigma_1],
[mu, sigma_2]])
sample_size = 1000
p1 = 0.2 # chance that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the element
GMM_idx = np.random.alternative(len(mixing_prob), dimension=sample_size, exchange=True,
p=mixing_prob)
# GMM_data is the GMM pattern knowledge
GMM_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx),
dtype=np.float64)Then we plot the histogram of the generated knowledge versus the chance density operate as proven under. The determine reveals the contribution of each Gaussian densities within the general GMM, with every density scaled by its corresponding issue.
The Python code used to generate the above determine is proven under.
x1 = np.linspace(GMM_data.min(), GMM_data.max(), sample_size)
y1 = np.zeros_like(x1)
# GMM chance distribution
for (l, s), w in zip(norm_params, mixing_prob):
y1 += stats.norm.pdf(x1, loc=l, scale=s) * w
# Plot the GMM chance distribution versus the information histogram
fig1, ax = plt.subplots()
ax.hist(GMM_data, bins=50, density=True, label="GMM knowledge histogram",
coloration = GRAY9)
ax.plot(x1, p1*stats.norm(loc=mu, scale=sigma_1).pdf(x1),
label="p1 × first PDF",coloration = GREEN1,linewidth=3.0)
ax.plot(x1, (1-p1)*stats.norm(loc=mu, scale=sigma_2).pdf(x1),
label="(1-p1) × second PDF",coloration = ORANGE1,linewidth=3.0)
ax.plot(x1, y1, label="GMM distribution (PDF)",coloration = BLUE2,linewidth=3.0)
ax.set_title("Information histogram vs. true distribution", fontsize=14, loc="left")
ax.set_xlabel('Information worth')
ax.set_ylabel('Chance')
ax.legend()
ax.grid()After that, we compute the estimate of the blending parameter p1 that we derived earlier utilizing MoM and which is proven right here once more under for reference.
The Python code used to compute the above equation utilizing our GMM pattern knowledge is proven under.
# Estimate the blending parameter p1 from the pattern knowledge utilizing MoM estimator
p1_hat = (sum(pow(x,2) for x in GMM_data) / len(GMM_data) - pow(sigma_2,2))
/(pow(sigma_1,2) - pow(sigma_2,2))In an effort to correctly assess this estimator, we use Monte Carlo simulation by producing a number of realizations of the GMM knowledge and estimate p1 for every realization as proven within the Python code under.
# Monte Carlo simulation of the MoM estimator
num_monte_carlo_iterations = 500
p1_est = np.zeros((num_monte_carlo_iterations,1))
sample_size = 1000
p1 = 0.2 # chance that the information level comes from first gaussian
mixing_prob = [p1, (1-p1)]
# A stream of indices from which to decide on the element
GMM_idx = np.random.alternative(len(mixing_prob), dimension=sample_size, exchange=True,
p=mixing_prob)
for iteration in vary(num_monte_carlo_iterations):
sample_data = np.fromiter((stats.norm.rvs(*(norm_params[i])) for i in GMM_idx))
p1_est[iteration] = (sum(pow(x,2) for x in sample_data)/len(sample_data)
- pow(sigma_2,2))/(pow(sigma_1,2) - pow(sigma_2,2))Then, we examine for the bias and variance of our estimator and evaluate to the theoretical outcomes that we derived earlier as proven under.
p1_est_mean = np.imply(p1_est)
p1_est_var = np.sum((p1_est-p1_est_mean)**2)/num_monte_carlo_iterations
p1_theoritical_var_num = 3*p1*pow(sigma_1,4) + 3*(1-p1)*pow(sigma_2,4)
- pow(p1*pow(sigma_1,2) + (1-p1)*pow(sigma_2,2),2)
p1_theoritical_var_den = sample_size*pow(sigma_1**2-sigma_2**2,2)
p1_theoritical_var = p1_theoritical_var_num/p1_theoritical_var_den
print('Pattern variance of MoM estimator of p1 = %.6f' % p1_est_var)
print('Theoretical variance of MoM estimator of p1 = %.6f' % p1_theoritical_var)
print('Imply of MoM estimator of p1 = %.6f' % p1_est_mean)
# Under are the outcomes of the above code
Pattern variance of MoM estimator of p1 = 0.001876
Theoretical variance of MoM estimator of p1 = 0.001897
Imply of MoM estimator of p1 = 0.205141We will observe from the above outcomes that the imply of the p1 estimate equals 0.2051 which could be very near the true parameter p1 = 0.2. This imply will get even nearer to the true parameter because the pattern dimension will increase. Thus, we now have numerically proven that the estimator is unbiased as confirmed by the theoretical outcomes completed earlier.
Furthermore, the pattern variance of the p1 estimator (0.001876) is sort of equivalent to the theoretical variance (0.001897) which is gorgeous.
It’s at all times a cheerful second when principle matches apply!
All photographs on this article, until in any other case famous, are by the creator.
Source link
