1.

Introduction

Square Kilometer Array (SKA)-Low^1,2 is the low-frequency part of the SKA telescope³ and is designed to detect radio signals in the frequency range of 50 to 350 MHz. It will be vastly more sensitive than existing instruments and powerful enough to detect faint signals from the dawn of the universe.⁴ Located in outback Western Australia, SKA-Low is an interferometer that will consist of an array of 131,072 dual-polarization log-periodic antennas, grouped in 512 electronically steered stations, each with 256 antennas. Every station in the array is composed of 16 tiles, each of which is a group of 16 combined antennas. A number of these antenna stations will be placed in a group at the center of the telescope (core) and the rest will span out along three spiral arms stretching $\sim 65\mathrm{km}$ end to end. Spreading the antennas over huge distances will enable very fine resolution imaging, revealing the universe in a level of detail never seen before. The unprecedented sensitivity achieved by the number of array elements, their collecting area, and the wide processed bandwidth is the basis for the development of many science cases being covered by SKA-Low.

Observation of faint sources requires integration time that can last for several minutes, making the sampling clock stability in different timescales to be relevant for the system performance. A stable clock is essential both for interferometric coherence and to achieve a high signal-to-noise ratio (SNR). The critical parameter for clock stability is the jitter of the analog-to-digital converter (ADC) sampling clock, defined as the difference between the ideal and the true time at which the analog input is sampled. Clock jitter directly translates into phase noise, and the related spectrum can be analyzed in the frequency domain. Phase noise can be represented as an added noise to the ideal sampling tone. For a typical clock signal, most of the noise power is clustered around the tone frequency, although a significant part of the power consists of a noise floor extending to most of the input signal bandwidth. As the sampling operation is basically a multiplication of the input signal by the sampling pulse, the presence of the jitter (and consequently phase noise) can be represented in the frequency domain as a convolution of the signal with the spectrum of the sampling clock. The result is that clock jitter modulates any strong RF signal, scattering part of its power on nearby spectral regions, thus reducing SNR. With respect to interferometric sensitivity, low-frequency clock jitter contributes to decorrelation causing coherence loss. Given the importance of frequency stability on the telescope sensitivity, it is fundamental to accurately design the system section dedicated to the generation, dissemination, and delivery of signals that support the sky signal digitization. In the low-frequency aperture array (LFAA), a reference frequency signal is distributed and delivered to the digital part of the telescope, namely the signal processing subsystem (SPS), with the purpose of generating the sampling clock for digitization. The nominal value for the reference frequency is 10 MHz.

Timing accuracy is another crucial part of SKA-Low as it affects the capability of the telescope to do precise time stamping of signals from astrophysical sources and ensuring synchronization of the system. Timing is provided by the pulse per second (PPS) signal which is the heartbeat of the telescope. PPS is locked to the Universal Time Coordinated (UTC), so it is both the realization of the base unit of time (second) and the representation of absolute time required to be known with very low uncertainty. This signal is also used for the synchronization of the overall SPS operation.

The synchronization and timing⁵ (SAT) network has to face the very challenging requirement to synchronize such a huge number of antennas, spread over very long distances, to within a stability level such that the array operates with good visibility and coherence. The SKA-Low solution for long-distance frequency synchronization uses a procedure called phase conjugation, sending a laser at a reference frequency (2 GHz) to the remote station, and a second laser sent back on the same fiber at 1 GHz. The technique employs a servo in the remote station to equalize the phase of the 2-GHz signal which has traveled one-way versus the 1-GHz signal which has traveled the round trip.⁶ The SKA-Mid solution uses a technique in which an 8-GHz microwave frequency reference is encoded as a difference frequency on optical fiber using a single-sideband (SSB) modulation of a highly coherent laser. At the antenna, the light-wave is slightly frequency-shifted and reflected by a Faraday mirror. The round-trip phase correction employs an acousto-optic modulator (fiber frequency shifter) device that closes the phase-loop by use of a low-frequency (40 MHz) voltage-controlled oscillator (VCO).⁷ The UTC time reference (both for low and mid) makes use of a publicly available hardware and software designs from a collaborative project called White Rabbit⁸ which is able to perform timing dissemination with sub-ns accuracy using mostly commercial off-the-shelf components and subsystems.⁹ Note that the White Rabbit technology, started to provide timing systems for particle accelerators, is evolving to a more general solution to be used on metrology institutes, Fintech data centers, radar applications, and astrophysics facilities.¹⁰

The Expanded Very Large Array¹¹ frequency synchronization system uses a centrally generated and distributed reference at 512 MHz, a round-trip phase measured at low bandwidth and open-loop as a phase accumulated on the same 512 MHz carrier, with the outgoing and returned signal on separate fibers. At the antenna, the reference is multiplied in microwave phase-locked loop (PLL) synthesisers to supply first and second local oscillator (LO).^12,13

In the Atacama Large Millimeter/Submillimeter Array¹⁴ Observatory, the tight frequency stability requirement is achieved with low phase noise crystal oscillator (XO) references, a low phase noise microwave reference common to all antennas, a tightly phase locked dual-laser system for generating the LO reference up to 122 GHz, a yttrium iron garnet—oscillator-warm-multiplier assembly at the antenna as a cleanup oscillator, and finally a cold multiplier to reach the highest millimeter wave frequency bands. To achieve ultralow phase drift, a round-trip correction system is implemented based on a stabilized single-frequency laser. The fiber length is stabilized with fiber stretchers, moreover, a polarization stability calibration and passive thermal stabilization are added to further improve the performance.^15,16

In addition, actively stabilized remote frequency dissemination is also adopted for synchronizing what are currently individually referenced very long baseline interferometry (VLBI) antennas. Medicina Radio Telescope,¹⁷ part of the European VLBI Network (EVN), is connected via a 550-km optical fiber link to the Italian National Metrological Institute (INRIM), where an ultrastable laser frequency-referenced to the primary standard (hydrogen maser) is used as a transfer oscillator and phase stabilization is realized by means of acousto-optic modulation. At the radio telescope, an RF signal is generated from the laser using an optical frequency comb.¹⁸ The Torun VLBI station¹⁹ in Poland is remotely synchronized to an atomic clock through 350 km of fiber over the Polish fiber-optic network for time and frequency (T&F) distribution (OPTIME).²⁰ This system uses an integrated circuit to perform delay compensation to the stabilized 10 MHz and synchronizes both time and frequency.²¹

The LFAA SPS physical implementation is based on the hierarchical structure shown in Fig. 1, where the elements composing the system are illustrated along with their quantities to better convey the dimension of the whole architecture.

Fig. 1

Hierarchical structure of the SPS physical implementation. Each tile (16 antennas) is digitized and processed in a tile processing module (TPM), eight TPMs are housed in a subrack, and four subracks (32 TPMs) are hosted in a cabinet.

In this paper, we only focus on the distribution network of the time and frequency reference signals inside the SPS cabinet (from the cabinet input to the TPMs). This work does not address the generation and delivering of standard T&F references to the input of SPS cabinets. In Sec. 2, we present the distribution architecture proposed by Italian Institute of Astrophysics (INAF) with the collaboration of Sanitas EG. Section 3 describes the hardware setup of the test bench used for the clock jitter measurements. Measurements of the low-frequency jitter in the 10-MHz distribution tree and of the high-frequency jitter in the captured signal are shown in Sec. 3.1 and Sec. 3.2, respectively. Finally, Sec. 4 concludes the discussion and addresses some last considerations.

2.

Clock and PPS Distribution Architecture Inside LFAA Cabinet

The design of the distribution chain is based on the assumption that each LFAA cabinet receives the 10-MHz frequency reference and the PPS from the Signal and Data Transport²² (SaDT) infrastructure.²³ These two signals have to be distributed to each TPM in the cabinet where analog-to-digital conversion takes place. Each ADC of the TPM shall receive a very stable sampling clock with minimum addition of phase noise not to corrupt the sky signal spectrum at digitization. The acquisition shall start synchronously for each ADC, with the PPS as the common time reference. ADC sampling clock and FPGA internal clock must be phase aligned with the distributed 10-MHz reference signal. In brief, the main purpose of the designed network is to provide the ADC synchronization over the cabinet, without substantially degrading the quality of the clock signal.

The design of the electronics devoted to the 10-MHz signal distribution has been conducted taking into account considerations of electromagnetic compatibility (EMC), limiting as much as possible radio-frequency interference (RFI) it can cause in the telescope bandwidth due to harmonics, and also avoiding susceptibility to noise sources such as high-speed digital circuits.

Another aspect considered for the current design is the complexity of the network and the consequent related cost.

From system design specifications, no information is available about the phase relation between incoming 10 MHz and PPS as they have different distribution paths. This relation may change even between individual boards inside a cabinet, due to difference in the PPS distribution paths among FPGAs at subrack or cabinet level, but is relatively stable over time. The TPM FPGA samples the PPS with a clock derived from the 10-MHz reference (see Sec. 2.4). This resampled signal is used to synchronize the acquisition and to derive the timestamping for individual samples. Timestamping is determined by counting samples from a fiducial sample, which is identified using the PPS signal. Cycle ambiguity may occur if the PPS timing does not satisfy setup and hold requirements of the resampling circuit, causing the timestamping to be different at different times in an unpredictable way. Although relative timing among antennas and stations can be determined by astronomic observation, it is essential that the relation between PPS and samples timestamp is stable in time. To avoid this possible event, the proposed solution is to measure the PPS phase with respect to the sampling clock at each FPGA, and to adjust the phase of the resampling clock to ensure a stable capture. The relative phase of the PPS and 10-MHz signals is also measured at the cabinet or subrack input and a programmable delay on the PPS signal itself can be introduced before the distribution. All these adjustments are performed in software, by measuring the relevant quantities, determining the appropriate offsets for each FPGA, and applying them. The applied offsets are then kept constant and readjusted only when necessary. The added offset information is recorded and will be later used to properly correct the acquired data timing. This scheme works if the drift in relative phase stability between the two signals is less than the period of the resampling clock (5 ns) minus the setup-hold avoidance window. SKA specifications require a stability of the PPS signal to at most 1 ns, which is more than sufficient. Experience with the AAVS array operation has shown that this is indeed the case.

The proposed architecture is a distribution tree implemented on three levels: cabinet, subrack, and TPM (see Fig. 2). The starting point is the cabinet management board that distributes the 10 MHz and PPS to four subracks. The second stage is represented by the subrack management board (SMB) and backplane which supply the two signals to the eight TPMs inside the subrack. Lastly, the sampling clock is generated and delivered to 16 ADCs in the TPM.

Fig. 2

SPS cabinet distribution tree organized in three levels: (a) cabinet, (b) subrack, and (c) TPM.

The adopted design solution is a mixed active/passive distribution structure with a cascade of three PLL circuits,²⁴ one per level. These electronic components are largely employed in applications requiring low-jitter clocks as they present excellent phase noise performance at limited costs. In our design, PLL circuits are used for the reference clock distribution and to measure the PPS phase delay, whereas the PPS is distributed by high-speed buffers. Clock distribution circuit is based on low-voltage differential signals, whereas the PPS distribution uses CMOS signals that support 5 V input voltage. The diagram of Fig. 3 shows the hierarchy and components responsible for phase noise in the clock tree (upper part), along with the distribution path of the PPS signal (lower part).

Fig. 3

Overview of the hierarchy of the clock and PPS distribution inside the cabinet.

In the next section, we introduce the selected PLL for the first two distribution stages (at cabinet and subrack levels): the AD9545 Digital PLL,²⁵ released by Analog Devices, Inc. (ADI) in 2017.

2.1.

Analog Devices AD9545 Digital PLL

The core of the device comprises two independent PLLs operating in parallel. Each PLL is actually the assembly of a digital and an analog PLL connected in series as pointed out in Fig. 4.

Fig. 4

Fundamental building blocks of the AD9545 Digital PLL.

Among its many features (for details see Ref. 25), the DPLL portion of each PLL channel includes a programmable digital loop filter with extremely low loop bandwidth capability that enables a significant reduction of jitter transfer from the reference input to the output.

The system clock is the primary time base of the AD9545 for all of its time-keeping functions. The device synthesizes its system clock via a system clock PLL, which upconverts the reference signal applied to the system clock input pins.

We have extensively tested this highly configurable component using the evaluation board and the related ADI software that easily permits to program the registers. Tests have been carried out in a laboratory at the Medicina Radio Astronomical Station (Bologna, Italy) utilizing the time and frequency reference signals available at the observatory. Figure 5 shows the laboratory test setup for the AD9545 evaluation board. With the technical support of ADI engineers, we have achieved the correct configuration for our application. An interesting feature deserves to be mentioned. After some tuning of the device configuration, we have been able to measure the phase offset between the two input signals, i.e., 10 MHz and PPS. The user time stamp processors provide access to time stamps from various sources on the device. They work in conjunction with auxiliary numerically controlled oscillator, which provides the reference time for reporting user time stamps. Observing time stamps, the user has phase (referred to common reference) and frequency information of individual inputs so, with a proper subtraction, it is possible to deduce the phase offset between them. We have performed such a measurement and the results have been confirmed and validated executing the same measurement with the oscilloscope.

Fig. 5

AD9545 evaluation board tested in laboratory.

Note that from this point forward we refer to the AD9545 device using the acronym DPLL for the sake of simplicity.

2.2.

Management Board

The management board is designed to provide the functionality both at cabinet and subrack levels, depending on two different configurations, with the same architecture and layout. The main functions of the board are shown in Fig. 6, while a picture of the board is shown in Fig. 7. In the cabinet configuration, the management board is boxed in a 1U drawer to interface with the system time and frequency reference signals (10 MHz and PPS) and distribute them to all subracks inside the cabinet. In the subrack configuration, the management board is plugged in the backplane for the distribution to all TPMs of the subrack. The board size is $233\times 350\mathrm{mm}$ and it has been defined to be compatible with TPM layout and to fit the subrack dimensions, as specified in Ref. 26. For EMC, the board layout is arranged to separate the clock areas from potential RFI culprits such as high-speed digital circuits.

Fig. 6

Graphic representation of management board layout (top view) showing the placement of the main functions (yellow boxes).

Fig. 7

Picture of the management board.

The clock and PPS distribution section includes a differential $\times 4$ RF clock buffer, the DPLL (previously described in Sec. 2.1), a digital clock buffer, and a selection of 49.152 MHz high-quality system clock oscillators. The DPLL is used to regenerate the 10-MHz reference clock removing the high-frequency noise components from the input reference signal. A low-pass digital filter is enabled on the input and only the phase reference information is maintained in the generated output. Filtering spurious high-frequency signal components from the incoming reference clock greatly reduces the possibility of higher frequency electromagnetic emissions from the cables in the distribution chain.

The design of the management board implements a passive PPS distribution, with the capability to measure and manage the phase relation between the input 10 MHz reference and PPS signal.

2.2.1.

Cabinet configuration

The cabinet configuration structure of the management board is shown in the block diagram of Fig. 8. The input 10-MHz reference signal is connected to the DPLL to generate a phase aligned 10- and 100-MHz outputs. These signals are provided to a complex programmable logic device (CPLD) together with the PPS signal to carry out a rough measurement of the PPS input phase. One 10-MHz output (after jitter cleaning) is routed to the $4\times$ clock buffer that generates four synchronous signals, one for each subrack. Before each clock output connector, a low-pass filter further removes the high-frequency spectral components of the signal. Besides the CPLD, the PPS signal is also connected to the DPLL. Another 10-MHz output signal (in phase with the input) is routed back to the DPLL to execute a precise phase offset (time difference) measurement between the PPS rising edge and the 10-MHz closest rising edge. As previously mentioned, the CPLD receives the PPS and, if needed, performs a programmable delay line to change the PPS output phase before it is distributed to the subracks with the dedicated clock buffer. The measurement of the time offset between the 10 MHz and PPS at the beginning of the acquisition and the possibility to adjust the PPS phase is crucial to avoid possible synchronization issues among ADCs and allow a stable acquisition.

Fig. 8

Block diagram with the main functional components of the management board in the cabinet configuration. Thin red lines are the interconnections for the clock distribution, while yellow lines refer to the PPS.

2.2.2.

Subrack configuration

The clock distribution architecture of the subrack configuration is similar to the implementation for the cabinet case, and the only difference is that two 10-MHz outputs from the DPLL are directly routed to the backplane. The PPS, after it is regenerated and possibly adjusted in phase by the CPLD, is connected to the backplane with two signals through the high-density backplane I/O connector. The PPS and clock section of the subrack configuration is included in Fig. 9. If compared to the previous Fig. 8, it is clear the difference from the cabinet configuration.

Fig. 9

Block diagram with the main functional elements of the management board in the subrack configuration. Thin red lines are the interconnections for the clock distribution, while yellow lines refer to the PPS.

2.3.

Backplane

Backplane is situated at the rear of the subrack chassis and provides each TPM with power and services required for LFAA operations. It features the mechanical interface for the connection with 8 TPMs and 1 SMB. One of the main functions of the backplane is to supply a passive distribution of the reference and synchronization signals to the TPM boards inside the subrack. More specifically, it distributes the PPS (two single-ended signals) coming from the SMB to each TPM. It also performs the distribution of two 10-MHz clock signals with differential interface. The distribution paths of PPS and clock have the same length, and they are equal for every TPM. Furthermore, they have low insertion loss and an isolation versus all other signals $>60\mathrm{dB}$. In Fig. 10, it is presented a schematic drawing of the backplane (green) and the related interfaces with the other components of the subrack.

Fig. 10

Backplane block diagram displaying the main interconnections and functions. Note the distribution route of PPS (violet) and 10 MHz (red) from the SMB to the TPMs.

2.4.

Tile Processing Module

The TPM^27,28 is the main processing component within the SKA-LFAA element.²⁹ It is a hybrid analog/digital platform that receives 32 analog input signals coming from 16 dual polarization antennas (one SKA-Low tile) via RF over fiber links and contributes with the cooperation of other tile processors to form the station beam.³⁰ The TPM is an assembly mainly composed of the analog-to-digital unit (ADU) board and two pre-ADU boards. Pre-ADU boards are devoted to optical–electrical conversion, filtering, amplification, and equalization of the analog signals. The ADU board has the task of analog-to-digital conversion and subsequent signal processing in the FPGA devices.

The 10-MHz clock signal is supplied as the phase reference to the PLL (Analog Devices AD9528) of the ADU board that generates all the necessary signals for the data acquisition and processing (see Fig. 11). The TPM internal synchronization is based on the JESD204B standard,³¹ which provides the interface between the data converter (ADC) and the logic device (FPGA) and regulates all the acquisition operations. The PLL generates two sampling clocks, one can run at 700, 800, or 1000 MHz, whereas the other runs at a lower speed. The latter is divided by a power of two in the FPGA and used for the data processing. It shall be noted that the FPGA input clock buffer and internal PLL can receive a maximum external clock frequency below the required ADC sampling frequency. One of the two FPGAs receives the digital PPS signal which is sampled with a clock phase related to the 10-MHz reference. The FPGA samples the PPS with a clock that is generated internally multiplying the data processing clock. The FPGA logic consequently manages the PPS and the acquisition logic to start the acquisition on the specific multiplied clock edge, with a deterministic and stable phase offset with respect to the 10-MHz reference. The two PLL outputs are each routed to a $\times 8$ clock buffer (Analog Devices ADCLK948) with high-quality differential traces and then distributed to all the ADC clock inputs. The JESD SYSREF signals, required to properly start the acquisition, are provided in input to the PLL and distributed with the same circuitry (not shown in Fig. 11). With this scheme, the TPM related jitter noise comes from the combination of the PLL and the clock buffer distribution operations. Similar to other PLLs in previous stages of the chain, this PLL also operates as jitter cleaner on the input 10-MHz reference which is used to phase align a first 100 MHz generated with an on board voltage-controlled crystal oscillator. This tone is then multiplied by a second internal PLL circuit to an higher frequency that is later divided on the output buffers to provide the required frequencies. The TPM clock distribution circuit is almost symmetrical with respect to the vertical axis of the board layout, with the main PLL placed in the center of the board across the digital and the RF area. All the ADC input signals have the same route scheme and are almost identical. This approach has been adopted to minimize possible ADC performance differences across the board, between the edge and center position.

Fig. 11

Clock distribution scheme in the TPM board. A PLL generates the ADC sampling clock and the clock used by FPGA logic for data acquisition and processing. These clocks are phase aligned with the input 10 MHz reference. PPS is sampled by FPGAs for time synchronization.

Antenna signals are digitized using an Analog Devices AD9695 ADC, which has a resolution of 14 bits. However, in standard telescope operation only the eight most significant bits are actually sent to the processing FPGA, due to bandwidth limitation.

3.

Clock Jitter Measurement

SKA jitter specifications are related to two high-level specifications. The clock jitter must not cause a decorrelation loss greater than 2% on timescales of 1 to 60 s, and the added noise due to jitter must not exceed 10% of the quantization noise. Both these specifications depend on signal frequency so we considered the highest observing frequency of 350 MHz. The first requirement translates to a maximum jitter of 70 ps. The second is dependent on the RFI level, as the most challenging situation occurs when a strong interference dominates the acquired signal power. We assumed an RFI power equal to the maximum ADC power level before ADC saturation, i.e. around 70 quantization units RMS. The resulting maximum jitter is 0.6 ps RMS. As RFI levels above 40 quantization units produce considerable nonlinearities, higher jitter levels, up to 1 ps, could be acceptable. It should be noted that these specifications relate to very different frequency regions in the phase noise spectrum. The first is important at frequencies below a few kHz, while for the second only the phase noise above 1 kHz is relevant. We have not analyzed long-term stability for times greater than 1 s, usually specified as an Allan variance, as it is dominated by the global clock distribution network, which is outside the scope of this work.

We have measured the clock jitter contribution both in the 10-MHz distribution tree up to the TPM input port, and at the ADC sampler by analyzing the effects on a digitized signal. First, we have measured the phase noise spectrum of the distributed clock or of the digitized signal, in the frequency domain, then we have translated it to a time-domain jitter, separating the contribution of different frequency intervals. SSB phase noise $\mathcal{L}(f)$ is defined as the ratio of the noise in a 1-Hz bandwidth at a specified frequency offset from the carrier frequency $f_c$ and the carrier power itself. The power ratio is typically expressed in logarithmic scale, dBc/Hz, and is plotted as a function of frequency offset with the frequency axis also represented on a log scale. The RMS time-domain jitter in a sinusoidal signal, either the clock reference or a digitized tone, can be directly computed from the associated phase noise spectrum $\mathcal{L}(f)$ (in dBc/Hz) with the following equation:³²

In the following sections, we describe these measurements, performed in the laboratories of the Medicina observatory between June, 2019, and March, 2020.

3.1.

Jitter in the Clock Distribution Tree

Jitter introduced by the first two stages of clock distribution and the subrack backplane has been measured using the instrumental configuration and the actual setup shown in Figs. 12 and 13, respectively. The input clock signal is a highly stable 10-MHz sinusoidal tone, derived from the Medicina telescope Hydrogen Maser (H-maser) master clock. We have initially used a Tektronix MSO 70404C Mixed Signal Oscilloscope equipped with DPOJET application software, an analysis tool for advanced jitter and timing measurement on clocks and data signals. Then, we have opted to perform the measurements using a Rohde and Schwarz FSWP50 phase noise analyzer, that features extremely high sensitivity for phase noise measurements due to cross-correlation technique (typ. $-172\mathrm{dBc}/\mathrm{Hz}$ at 1 GHz and 10 kHz offset). The analysis was carried out on 1 Hz to 3 MHz bandwidth. RMS jitter has been calculated from phase noise, on the whole band or on a specific sub-band.

Fig. 12

Architectural block diagram of the 10-MHz reference frequency distributed at cabinet and subrack level. The output of the chain is tested with an oscilloscope and a phase noise analyzer.

Fig. 13

Measurement setup of the 10-MHz clock signal distributed through the chain composed of cabinet management board, SMB and backplane board: (a) the circuit under test is in the middle, its output is connected to the oscilloscope (left) and a PC (right) controls the board registers and (b) R&S FSWP50 phase noise analyzer.

Measured SSB phase noise spectra of various signals and instrumental setups are shown in Fig. 14. The input reference clock itself has been measured first (lower blue line). Phase noise spectrum is relatively flat, and the total jitter is around 750 fs, dominated (700 fs) by the high-frequency portion of the spectrum. The noise at the DPLL output is considerably higher. We have tested two different oscillators to provide the DPLL with the necessary input system clock: a XO and an oven-controlled crystal oscillator (OCXO). The phase noise has been measured both at the output of the cabinet management board and at the output of the backplane. The resulting spectra are similar for these configurations and are presented in the central group of four plots in Fig. 14. The phase noise spectrum is basically divided into two parts. For $f<100\mathrm{Hz}$, the phase noise follows the input signal, but it adds a significant $1/f^2$ noise (white frequency noise). This component is likely related to noise in the VCO control voltage of the DPLL, which can be generated by the DPLL phase comparator. The displayed phase noise spectrum corresponds to a fractional frequency jitter of $\mathrm{\Delta }f/f={10}^{-11}$ in 1 s, against a fractional frequency jitter of $\mathrm{\Delta }f/f={10}^{-12}$ in the reference clock. Clock jitter due to this component is $\approx 3\mathrm{ps}$, 1.7 ps for each DPLL in the chain. For $f>1\mathrm{kHz}$, spectrum is almost flat in phase, with a total jitter of 2.5 ps (1 kHz to 10 MHz excluding harmonics).

Fig. 14

Measured SSB phase noise of 10-MHz reference tested at different points of the distribution chain and with different configurations. It is also compared to the input H-maser source and to the output of the Octoclock.

For comparison, two different configurations have been also measured. The distribution system used in the prototype array, an Octoclock 8 channels clock distribution module, performs better at low frequency, but has a higher noise above 1 kHz. Using a much worse input reference clock (red line), we have tested the cleaning performance of the two DPLLs for frequencies above 100 Hz (green line). A phase noise rejection better than 50 dB in this frequency region is apparent from the plot.

The phase noise for the complete chain, using the crystal oscillator in both boards, has been measured with very long integration times, to characterize the spectral region below 1 Hz. The complete spectrum is shown in Fig. 15. The excess noise due to the DPLL phase detector can be seen between 0.1 and 100 Hz. On longer timescales, the phase comparator limits the total phase error, and the noise converges back to the reference noise spectrum. Below 100 mHz, the spectrum follows the reference clock phase spectrum, with a $1/f^2$ slope up to 1 mHz.

Fig. 15

Measured SSB phase noise of 10-MHz reference distributed at cabinet and subrack level. Both DPLLs are configured with the crystal oscillator as the source for the system clock input. Note the starting offset frequency of 1 mHz.

3.2.

Jitter at the ADC Sampler

The total jitter introduced by the clock distribution network, the ADC clock generation in the TPM, and the ADC itself can be measured by analyzing the jitter in a monochromatic tone, digitized by the complete system. It is important to disentangle the contribution of the clock jitter from the intrinsic jitter in the acquired tone and from the ADC quantization noise. The test hardware setup is depicted in Fig. 16. The ultrastable 10-MHz reference signal of the H-Maser, available at Medicina premises, is supplied to a TPM board with the distribution chain described in Sec. 2, consisting of cabinet management board, SMB, and backplane. In the TPM, the signal is upconverted by the internal PLL to generate an 800-MHz sampling clock which is delivered to each ADC. A Rohde and Schwarz SMA100B signal generator, equipped with the ultralow phase noise option and locked to the H-Maser, provides a stable tone with relatively (compared with the device-under-test) low additive phase noise. The harmonic performance of the generator is improved by filtering the analog signal with a band-pass filter before it is injected in the ADC input. The generator output power is set to 2 dB below the ADC full scale, i.e., 10 dBm or 70 ADC units RMS amplitude.

Fig. 16

Test setup for the total jitter measurement. Distributed clock is fed to TPM as the reference signal for the internal PLL. A high-purity tone is digitized and the resulting samples are transmitted to a PC for data analysis. The signal generator is locked on the 10-MHz H-maser source.

The standard 8 bit acquisition mode of the TPM introduces a quantization noise around $-130\mathrm{dBc}/\mathrm{Hz}$, and therefore a new FPGA firmware design has been developed to acquire 14 bits from one ADC channel and capture a block of contiguous time samples. These data are stored in the DDR memory of TPM board and then transmitted to a PC. Available DDR memory size limits the total acquisition time to 0.283 s, corresponding to 1023 frames, each composed of $256*864$ samples. Once downloaded and written on computer disk, signal samples are analyzed with Matlab^® scripts. Data processing includes apodized fast Fourier transform with Nuttall windowing (apodization of $>90\mathrm{dB}$ at 100 Hz offset, decreasing to $>130\mathrm{dB}$ at 1 kHz offset), time average over 13 power spectra, RF tone subtraction, and calculation of SSB phase noise. The resulting spectral resolution with this test setup is $\sim 24\mathrm{Hz}$, and the instrumental noise floor is around $-145\mathrm{dBc}/\mathrm{Hz}$.

Examples of phase noise spectra are shown in Fig. 17. Noise below 250 Hz is due to spectral leakage. The noise drops to basically the instrumental floor above 1 MHz. Harmonics in the high portion of the spectrum are due to ADC nonlinearities and contribute very little to the total noise power.

Fig. 17

Examples of measured SSB phase noise with different frequency value of the sampled RF signal: (a) $f=150\mathrm{MHz}$ and (b) $f=250\mathrm{MHz}$.

Equation (1) has been used to derive the signal jitter. The result of this measurement is actually a total jitter that includes both ADC aperture jitter and sampling clock jitter, whereas the contribution due to the signal generator can be considered negligible given its excellent performance in terms of SSB phase noise. The derived tone jitter, as a function of the tone frequency, is plotted in Fig. 18 for different choices of the integration window in Eq. (1). The lower frequency offset has been chosen as 240 Hz or 1 kHz, with a very small difference in the total jitter. The ADC quantization noise contributes significantly to the total measured noise spectrum, especially for low tone frequencies (blue line). The strong decrease in the computed jitter is consistent with the instrumental effect of a white quantization noise. As the true phase noise drops above 1 MHz, we have chosen an upper integration bound of 10 MHz to exclude most of the quantization noise contribution.

Fig. 18

Total jitter as a function of the tone frequency computed for an integration window of 240 Hz to 100 MHz (blue), 240 Hz to 10 MHz (orange), 1 kHz to 10 MHz (yellow), and corrected for ADC noise (purple).

The jitter is around 350 fs for most of the SKA-Low observing band, rising to 400 fs near the band edges. Assuming a flat quantization noise spectrum, we also subtracted this contribution from the total noise in Eq. (1), to determine an (optimistic) lower bound to the actual jitter of around 300 fs.

4.

Conclusion and Final Remarks

In this work, we analyzed the contribution of the SPS clock and timing distribution network on the SKA clock stability against the high-level SKA requirements, listed in Sec. 3. A strategy to obtain a deterministic and stable timing despite the undetermined phase relationship between these two signals has been outlined in Sec. 2.

The total contribution of the clock distribution tree to the jitter in the quantization process shows two different regimes. At very low frequency offsets, between 0.1 and 200 Hz, the dominant contribution is due to the PLLs in the two distribution boards. The total jitter is about 3 ps. This corresponds to a phase jitter of 7 mrad at the highest observing frequency, with a negligible decorrelation effect on both the station beamformer and the interferometric correlator. The PLLs provide a very strong rejection of any high-frequency phase noise in the reference clock. At higher-frequency offset, above the SKA-Low minimum channel separation of 226 Hz, the total jitter is below 400 fs. The corresponding spectral leakage of even the strongest RFI signals is at least one order of magnitude below the 8-bit ADC quantization noise.

This work does not analyze the effects of the global clock and timing distribution network, from the telescope reference to the individual SPS cabinets, both in the central and in the remote processing facilities; however, some important considerations can be drawn. Although no phase relationship is assumed between the 10-MHz reference and the PPS timing signal, the arbitrary phase between these signals at each cabinet input must be sufficiently stable (to $\pm 2\mathrm{ns}$) to guarantee a consistent capture of the PPS signal. The low frequency jitter, on timescales ranging from 10 ms to the longest integration times, is dominated by the SKA-Low global distribution network. The contribution of the SPS cabinet distribution tree is minor, especially for remote processing facilities. The high-frequency jitter, corresponding to phase noise frequencies above 200 Hz, is effectively filtered by the SPS distribution tree. The very tight requirement (jitter better than 0.6 ps RMS) for this jitter is always satisfied.