Image reconstruction in nuclear medicine produces valuable volumetric data of vital markers in living bodies. Visual scene reconstruction methods, that aim to recreate a scene from camera images, are also continuously improved by the recent advancements of light-fields and camera systems. The parallels of the two fields are increasingly noticeable as we now have the computing power and methods to take into account transparent materials and ray trace the scattering and other effects of lights for visual scene reconstruction. In this paper, we aim to highlight and analyze the similarities and potential synergies of the two methods.
Multi-camera networks are becoming ubiquitous in a variety of applications related to medical imaging, education, entertainment, autonomous vehicles, civil security, defense etc. The foremost task in deploying a multi-camera network is camera calibration, which usually involves introducing an object with known geometry into the scene. However, most of the aforementioned applications necessitate non-intrusive automatic camera calibration. To this end, a class of camera auto-calibration methods imposes constraints on the camera network rather than on the scene. In particular, the inclusion of stereo cameras in a multi-camera network is known to improve calibration accuracy and preserve scale. Yet most of the methods relying on stereo cameras use custom-made stereo pairs, and such stereo pairs can definitely be considered imperfect; while the baseline distance can be fixed, one cannot guarantee the optical axes of two cameras to be parallel in such cases. In this paper, we propose a characterization of the imperfections in those stereo pairs with the assumption that such imperfections are within a considerably small, reasonable deviation range from the ideal values. Once the imperfections are quantified, we use an auto-calibration method to calibrate a set of stereo cameras. We provide a comparison of these results with those obtained under parallel optical axes assumption. The paper also reports results obtained from the utilization of synthetic visual data.
Light-field visualization allows the users to freely choose a preferred location for observation within the display’s valid field of view. As such 3D visualization technology offers continuous motion parallax, the users location determines the perceived orientation of the visualized content, if we consider static objects and scenes. In case of interactive light-field visualization, the arbitrary rotation of content enables efficient orientation changes without the need for actual user movement. However, the preference of content orientation is a subjective matter, yet it is possible to be objectively managed and assessed as well. In this paper, we present a series of subjective tests we carried out on a real light-field display that addresses static content orientation preference. The state-of-the-art objective methodologies were used to evaluate the experimental setup and the content. We used the subjective results in order to develop our own objective metric for canonical orientation selection.
The recent advances in light-The recent advances in light-field acquisition and display systems bring closer the day when they become commercially available and accessible to wide audiences for numerous use cases. Their usefulness and potential benefits have already been disseminated in the field and they started emerging in both industry and entertainment applications. The long-term goal of the scientific community and future manufacturers is to research and develop fully immersive, yet seamless and efficient systems that can achieve the ultimate visual experience. However, certain paths leading to such goals are blocked by technological and physical limitations, and also significant challenges that have to be coped with. Although some issues that rise regarding the development of capture and display systems may actually be nearly impossible to overcome, the potential for light-field applications is indeed immense, thus worth the vast scientific effort. In this paper, we systematically analyze and present the current and future relevant limitations and challenges regarding the research and development of light-field systems. As current limitations are primarily application-specific, both challenges and potentials are approached from the angle of end-user applications. The paper separately highlights the use case scenarios for industry and entertainment, and for everyday commercial usage. Currently existing light-field systems are assessed and introduced from a technical perspective and also with regards to usability, and potential future systems are described based on state-of-art technologies and research focuses. Aspects of practical usage, such as scalability and price, are thoroughly detailed for both light-field capture and visualization.
Real-time video transmission services are unquestionably dominating the flow of data over the Internet, and their percentage of the global IP packet traffic is still continuously increasing. As novel visualization technologies are emerging, they tend to demand higher bandwidth requirements; they offer more visually, but in order to do so, they need more data to be transmitted. The research and development of the past decades in optical engineering enabled light-field displays to surface and appear in the industry and on the market, and light-field video services are already on the horizon. However, the data volumes of high-quality light-field contents can be immense, creating storing, coding and transmission challenges. If we consider the representation of light-field content as a series of 2D views, then for a single video frame, angular resolution determines the number of views within the field of view, and spatial resolution defines the 2D size of those views. In this paper, we present the results of an experiment carried out to investigate the perceptual differences between different angular and spatial resolution parametrization of a light-field video service. The study highlights how the two resolution values affect each other regarding perceived quality, and how the combined effects are detected, perceived and experienced by human observers. By achieving an understanding of the related visual phenomena, especially degradations that are unique for light-field visualization, the design and development of resource-efficient light-field video services and applications become more straightforward.
Medical images and videos are now increasingly part of modern telecommunication applications, including telemedicinal applications, favored by advancements in video compression and communication technologies. Medical video quality evaluation is essential for modern applications since compression and transmission processes often compromise the video quality. Several state-of-the-art video quality metrics used for quality evaluation assess the perceptual quality of the video. For a medical video, assessing quality in terms of “diagnostic” value rather than “perceptual” quality is more important. We present a diagnostic-quality–oriented video quality metric for quality evaluation of cardiac ultrasound videos. Cardiac ultrasound videos are characterized by rapid repetitive cardiac motions and distinct structural information characteristics that are explored by the proposed metric. Cardiac ultrasound video quality index, the proposed metric, is a full reference metric and uses the motion and edge information of the cardiac ultrasound video to evaluate the video quality. The metric was evaluated for its performance in approximating the quality of cardiac ultrasound videos by testing its correlation with the subjective scores of medical experts. The results of our tests showed that the metric has high correlation with medical expert opinions and in several cases outperforms the state-of-the-art video quality metrics considered in our tests.
The Image Library for Intelligent Detection Systems (i-LIDS) provides benchmark surveillance datasets for analytics systems. This paper proposes a methodology to investigate the effect of compression and frame-rate reduction, and to recommend an appropriate suite of degraded datasets for public release. The library consists of six scenarios, including Sterile Zone (SZ) and Parked Vehicle (PV), which are investigated using two different compression algorithms (H.264 and JPEG) and a number of detection systems. PV has higher spatio-temporal complexity than the SZ. Compression performance is dependent on scene content hence PV will require larger bit-streams in comparison with SZ, for any given distortion rate. The study includes both industry standard algorithms (for transmission) and CCTV recorders (for storage). CCTV recorders generally use proprietary formats, which may significantly affect the visual information. Encoding standards such as H.264 and JPEG use the Discrete Cosine Transform (DCT) technique, which introduces blocking artefacts. The H.264 compression algorithm follows a hybrid predictive coding approach to achieve high compression gains, exploiting both spatial and temporal redundancy. The highly predictive approach of H.264 may introduce more artefacts resulting in a greater effect on the performance of analytics systems than JPEG. The paper describes the two main components of the proposed methodology to measure the effect of degradation on analytics performance. Firstly, the standard tests, using the ‘f-measure’ to evaluate the performance on a range of degraded video sets. Secondly, the characterisation of the datasets, using quantification of scene features, defined using image processing techniques. This characterization permits an analysis of the points of failure introduced by the video degradation.
A technique to improve the rendering quality of novel views for colour plus depth based 3D video is proposed. Most
depth discontinuities occur around the edges of depth map objects. If information around edges of both colour and depth
map images is lost during transmission, this will affect the quality of the rendered views. Therefore this work proposes a
technique to categorize edge and surrounding areas into two different regions (Region Of Interests (ROIs)) and later
protect them separately to provide Unequal Error Protection (UEP) during transmission. In this way the most important
edge areas (vital for novel view rendering) will be more protected than other surrounding areas. This method is tested
over a H.264/AVC based simulcast encoding and transmission setup. The results show improved rendered quality with
the proposed ROI-based UEP method compared to Equal Error Protection (EEP) method.
In 3D video delivery, the rendered 3D video quality at the receiver-side can be affected by rendering artifacts as well as
by concealment errors which occur in the process of recovering missing 3D video packets. Therefore it is vital to have an
understanding of the artifacts prior to transmitting data. This work proposes a model to quantify rendering and
concealment errors at the sender-side and to use the information generated through the model to effectively deliver 3D
We present a framework for the analysis of frame synchronization based on Synchronization Words (SWs), where the detection is based on the common sequential algorithm: the received samples are observed over a window of length equal to the SW; over this window a metric (e.g. correlation) is computed; a SW is declared if the computed metric is greater than a proper threshold, otherwise the observation window is time-shifted of one sample. We assume a Gaussian channel, antipodal signalling and coherent detection, where soft values are provided to the frame synchronizer. We state the problem starting from the hypothesis testing theory, deriving the optimum metric (optimum likelihood ratio test (LRT)) according to the Neyman-Pearson lemma. When the data distribution is unknown, we design a simple and effective test based on the Generalized LRT (GLRT). %added - begin
We also analyze the performance of the commonly used correlation metric, both in the "hard" and "soft" version. We show that synchronization by correlation can be greatly improved by the LRT and GLRT metrics, and also that, among correlation based tests, sometimes hard correlation is better than soft correlation. The obtained closed form expressions allow the derivation of the receiver operating characteristic (ROC) curves for the LRT and GLRT synchronizers, showing a remarkable gain with respect to synchronization based on correlation metric. The effect on the performance of non-equally distributed data is also shown.