## 1.

## Introduction

The conventional method of calculating the bit error ratio (BER) using
$Q$
-factor analysis assumes that the statistics of the received current is Gaussian. Whereas for nonreturn to zero (NRZ)-type systems employing direct detection, the assumption of Gaussian statistics yields satisfactory results due to a fortunate cancellation of errors,^{1} phase-shift-keyed modulation and balanced detection show non-Gaussian receiver current statistics.^{2, 3, 4, 5, 6} Recently^{7} we showed that duobinary transmission over
$600\phantom{\rule{0.3em}{0ex}}\mathrm{km}$
of nonzero-dispersion-shifted fiber (NZDSF) also exhibits significant departures from Gaussian statistics through a comparison between experiment and simulations in a 38-channel 10.7-Gbits/s dense wavelength-division multiplexed (DWDM) system, wherein our numerical predictions of non-Gaussian statistics were correctly confirmed by experiment.

In this paper, we demonstrate that while uncompensated dispersion plays the dominant role in inducing departures from Gaussian statistics, duobinary signaling inherently exhibits non-Gaussian statistics due to its intrinsically strong intersymbol interference. This is especially true when duobinary is generated using a low-pass filter at the transmitter.^{7} We model an 8-channel 10.7-Gbits/s DWDM system over
$800\phantom{\rule{0.3em}{0ex}}\mathrm{km}$
of NZDSF with no in-line dispersion compensation and show that standard
$Q$
-factor analysis relying on Gaussian statistics can severely underpredict the true system performance, which can only be captured by taking into account non-Gaussian statistics.^{2, 3, 4, 5, 6} The results are important because there has been considerable recent interest^{8} in increasing system reach with no in-line compensation using a combination of duobinary modulation and uncompensated low-dispersion NZDSF; in such a scenario, Gaussian statistics yields erroneous results.

## 2.

## System Description

Figure 1 shows the basic DWDM setup. The system consisted of eight optical channels spaced by
$50\phantom{\rule{0.3em}{0ex}}\mathrm{GHz}$
ranging from 1547 to
$1550\phantom{\rule{0.3em}{0ex}}\mathrm{nm}$
, each carrying duobinary signals. The transmission medium of the link consisted of eight spans of
$100\phantom{\rule{0.3em}{0ex}}\mathrm{km}$
of
${\mathrm{Corning}}^{\circledR}$
${\mathrm{LEAF}}^{\circledR}$
optical fiber. The fiber dispersion is about
$4\phantom{\rule{0.3em}{0ex}}\mathrm{ps}\u2215\left(\mathrm{nm}\phantom{\rule{0.3em}{0ex}}\mathrm{km}\right)$
at
$1550\phantom{\rule{0.3em}{0ex}}\mathrm{nm}$
. The pseudorandom bit sequence (PRBS) had a length of
${2}^{9}-1$
. All amplifiers in the link were single-stage erbium doped fiber amplifiers (EDFAs). The average launch power into each channel was about
$-1\phantom{\rule{0.3em}{0ex}}\mathrm{dB}\phantom{\rule{0.2em}{0ex}}\mathrm{m}$
. At the transmitter, we used a low-pass fifth-order Bessel electrical filter with a half width at half maximum (HWHM) of
$2.5\phantom{\rule{0.3em}{0ex}}\mathrm{GHz}$
. At the receiver, we used third-order Bessel optical filter with a HWHM of
$15\phantom{\rule{0.3em}{0ex}}\mathrm{GHz}$
and a fifth-order Bessel electrical filter with a HWHM of
$6\phantom{\rule{0.3em}{0ex}}\mathrm{GHz}$
. To elucidate the impact of uncompensated dispersion, we introduced varying amounts of lumped precompensation that compensated for 0, 25, and 50% of the cumulative dispersion value at
$1550\phantom{\rule{0.3em}{0ex}}\mathrm{nm}$
. In our case, since the cumulative dispersion value over the transmission fiber was nominally around
$3200\phantom{\rule{0.3em}{0ex}}\mathrm{ps}\u2215\mathrm{nm}$
at
$1550\phantom{\rule{0.3em}{0ex}}\mathrm{nm}$
, we introduced lumped dispersion-compensating fiber (DCF) that compensated for 0, 800, and
$1600\phantom{\rule{0.3em}{0ex}}\mathrm{ps}\u2215\mathrm{nm}$
. We compared these cases against a reference system impaired only by amplified spontaneous emission (ASE) noise. The optical SNR (OSNR) in all the four cases was around
$21\phantom{\rule{0.3em}{0ex}}\mathrm{dB}$
. We estimated the
$Q$
-factor at the end of the transmission system using two methods. In the first method, we used an accurate BER estimator based on a Karhunen-Loève expansion technique^{2, 3, 4, 5, 6} that does not make any assumptions about the nature of the received current; this method has produced accurate results,^{4, 5, 6} especially for balanced differential-phase-shift-keyed (DPSK) detection. The calculated BER was then converted to
$Q$
-factor. In the second method, we realized that there was significant patterning or banding caused by the interplay between dispersion and different bit sequences. Consequently, we applied the analysis first laid out by Anderson and Lyle^{9} to take into account intersymbol interference. The method divides the received data into different bit patterns (rails) and then assumes that the received current for each of the different patterns obeys Gaussian statistics with different means and variances and computes the
$Q$
-factor due to each of these individual patterns. The method then computes the average
$Q$
-factor using the individual
$Q$
-factors.

## 3.

## Results and Discussion

We present the results of our analysis in Fig. 2 through a plot of
$Q$
(in 20-log decibels) of the worst channel as a function of accumulated dispersion (in picoseconds per nanometer). We see that the discrepancy between the predictions of Gaussian statistics and those from the accurate BER estimator depends strongly on the amount of accumulated dispersion. Whereas the difference between Gaussian and non-Gaussian statistics leads to a
$Q$
difference of about
$1.2\phantom{\rule{0.3em}{0ex}}\mathrm{dB}$
for a completely noise dominated system, the difference increases to about
$2.7\phantom{\rule{0.3em}{0ex}}\mathrm{dB}$
for a system with no dispersion compensation. The difference in the strongly uncompensated regime is particularly important because recent work has focused on precisely this regime in demonstrating^{8} the value of duobinary transmission over uncompensated NZDSF. Thus, use of Gaussian statistics would severely underpredict the reach advantage of this system. Note here that although we report the results of the worst channel in Fig. 2, the differences among the various channels are negligible because the relative difference in accumulated dispersion among the various channels is small. In a system having a larger number of channels (difference in dispersion amongst channels), the channel with the largest accumulated dispersion would suffer the worst penalty.

Note that even in the case of a completely ASE-dominated system, departures from Gaussian statistics are significant, leading to more than $1\phantom{\rule{0.3em}{0ex}}\mathrm{dB}$ discrepancy in system performance. This shows that duobinary transmission inherently exhibits non-Gaussian statistics—uncompensated dispersion further emphasizes departures from Gaussian statistics. Therefore, whereas strong non-Gaussian statistics is induced due to accumulated dispersion and would be observed in other modulation formats, e.g., NRZ/ return to zero (RZ) as well, the duobinary modulation format presents an interesting case for two reasons. First, due to the strong patterning inherent in duobinary, this modulation format exhibits departures from Gaussian statistics even in an ASE-limited system. Second, duobinary is a significant modulation format in dispersion-dominated systems because of its tolerance to dispersion. Consequently, duobinary arguably presents a more interesting test case of non-Gaussian statistics than other amplitude-shift-keyed (ASK) formats.

To examine the cause of the discrepancy between Gaussian and non-Gaussian statistics in this uncompensated dispersion regime, in Fig. 3, we display an eye diagram of the worst channel after $800\phantom{\rule{0.3em}{0ex}}\mathrm{km}$ of uncompensated transmission. Here we have purposely ignored the effects of fiber nonlinearity and ASE noise to emphasize dispersion effects. We see that there are distinct banded levels reflecting the strong interplay between the spectra associated with different duobinary bit sequences and the fiber dispersion. Thus, the received current shows non-Gaussian statistics.

We can see the departures from Gaussian statistics more directly through a comparison of the probability density functions (pdfs) of the receiver current obtained from a Gaussian assumption and those obtained from the accurate BER estimator in Fig. 4. In the main figure, we show the pdfs of the 1’s and 0’s received currents from Gaussian approximation (dashed) and accurate (non-Gaussian) calculation (solid); in the inset, we magnify the tails of the pdfs to focus on the region where the tails intersect and where the BER will be actually calculated. The figure shows that while the regions near the centroid of the pdfs calculated by both the methods are similar, the tails are dramatically different. Since the BER depends on the tails of the PDFs and not the main body of the PDFs, Gaussian statistics and $Q$ -factor analysis produce erroneous results.

## 4.

## Conclusions

Using simulations of eight-channel $10.7\phantom{\rule{0.3em}{0ex}}\mathrm{Gbits}\u2215\mathrm{s}$ duobinary transmission over $800\phantom{\rule{0.3em}{0ex}}\mathrm{km}$ of NZDSF with no dispersion compensation we demonstrated that uncompensated dispersion can induce strongly non-Gaussian statistics of the receiver current. We showed that the use of Gaussian approximation and $Q$ -factor analysis severely underpredicts the system performance.