In this paper, we present a novel approach for the adaptation of large images to small display sizes. As a recent
study suggests, most viewers prefer the loss of content over the insertion of deformations in the retargeting
process.1 Therefore, we combine the two image retargeting operators seam carving and cropping in order to
resize an image without manipulating the important objects in an image at all. First, seams are removed carefully
until a dynamic energy threshold is reached to prevent the creation of visible artifacts. Then, a cropping window
is selected in the image that has the smallest possible window size without having the removed energy rise above
a second dynamic threshold. As the number of removed seams and the size of the cropping window are not fix,
the process is repeated iteratively until the target size is reached. Our results show that by using this method,
more important content of an image can be included in the cropping window than in normal cropping. The
"squeezing" of objects which might occur in approaches based on warping or scaling is also prevented.
In order to display a high dynamic range (HDR) video on a regular low dynamic range (LDR) screen, it needs
to be tone mapped. A great number of tone mapping (TM) operators exist - most of them designed to tone
map one image at a time. Using them on each frame of an HDR video individually leads to flicker in the
resulting sequence. In our work, we analyze three tone mapping operators with respect to flicker. We propose
a criterion for the automatic detection of image flicker by analyzing the log average pixel brightness of the tone
mapped frame. Flicker is detected if the difference between the averages of two consecutive frames is larger
than a threshold derived from Stevens' power law. Fine-tuning of the threshold is done in a subjective study.
Additionally, we propose a generic method to reduce flicker as a post processing step. It is applicable to all tone
mapping operators. We begin by tone mapping a frame with the chosen operator. If the flicker detection reports
a visible variation in the frame's brightness, its brightness is adjusted. As a result, the brightness variation is
smoothed over several frames, becoming less disturbing.
In this paper, we propose a new method to adapt the resolution of images to the limited display resolution
of mobile devices. We use the seam carving technique to identify and remove less relevant content in images.
Seam carving achieves a high adaptation quality for landscape images and distortions caused by the removal of
seams are very low compared to other techniques like scaling or cropping. However, if an image depicts objects
with straight lines or regular patterns like buildings, the visual quality of the adapted images is much lower.
Errors caused by seam carving are especially obvious if straight lines become curved or disconnected. In order
to preserve straight lines, our algorithm applies line detection in addition to the normal energy function of seam
carving. The energy in the local neighborhood of the intersection point of a seam and a straight line is increased
to prevent other seams from removing adjacent pixels. We evaluate our improved seam carving algorithm and
compare the results with regular seam carving. In case of landscape images with no straight lines, traditional
seam carving and our enhanced approach lead to very similar results. However, in the case of objects with
straight lines, the quality of our results is significantly better.
We enhance an existing in-circuit, inline tester for printed circuit assemblies (PCA) by video-based automatic optical
inspection (Video-AOI). Our definition of video is that we continuously capture images of a moving PCA, such that each
PCA component is contained in multiple images, taken under varying viewing conditions like angle, time, camera settings
or lighting. This can then be exploited for an efficient detection of faults. The first part of our paper focuses on the
parameters of such a Video-AOI system and shows how they can be determined. In the second part, we introduce techniques
to capture and preprocess a video of a PCA, so that it can be used for inspection.
In this paper, we introduce our new visualization service which presents web pages and images on arbitrary devices with
differing display resolutions. We analyze the layout of a web page and simplify its structure and formatting rules. The
small screen of a mobile device is used much better this way. Our new image adaptation service combines several
techniques. In a first step, border regions which do not contain relevant semantic content are identified. Cropping is used
to remove these regions. Attention objects are identified in a second step. We use face detection, text detection and
contrast based saliency maps to identify these objects and combine them into a region of interest. Optionally, the seam
carving technique can be used to remove inner parts of an image. Additionally, we have developed a software tool to
validate, add, delete, or modify all automatically extracted data. This tool also simulates different mobile devices, so that
the user gets a feeling of how an adapted web page will look like. We have performed user studies to evaluate our web
and image adaptation approach. Questions regarding software ergonomics, quality of the adapted content, and perceived
benefit of the adaptation were asked.
A large number of recorded videos cannot be viewed on mobile devices (e.g., PDAs or mobile phones) due to
inappropriate screen resolutions or color depths of the displays. Recently, automatic transcoding algorithms have
been introduced which facilitate the playback of previously recorded videos on new devices. One major challenge
of transcoding is the preservation of the semantic content of the videos. Although much work was done on the
adaptation of the image resolution, color adaptation of videos has not been addressed in detail before. In this
paper, we present a novel color adaptation algorithm for videos which preserves the semantics. In our approach,
the color depth of a video is adapted to facilitate the playback of videos on mobile devices which support only
a limited number of different colors. We analyze our adaptation approach in the experimental results, visualize
adapted keyframes and illustrate, that we obtain a better quality and are able to recognize much more details
with our approach.
The recognition of human postures and gestures is considered to be highly relevant semantic information in videos and surveillance systems. We present a new three-step approach to classifying the posture or gesture of a person based on segmentation, classification, and aggregation. A background image is constructed from succeeding frames using motion compensation and shapes of people are segmented by comparing the background image with each frame. We use a modified curvature scale space (CSS) approach to classify a shape. But a major drawback to this approach is its poor representation of convex segments in shapes: Convex objects cannot be represented at all since there are no inflection points. We have extended the CSS approach to generate feature points for both the concave and convex segments of a shape. The key idea is to reflect each contour pixel and map the original shape to a second one whose curvature is the reverse: Strong convex segments in the original shape are mapped to concave segments in the second one and vice versa. For each shape a CSS image is generated whose feature points characterize the shape of a person very well. The last step aggregates the matching results. A transition matrix is defined that classifies possible transitions between adjacent frames, e.g. a person who is sitting on a chair in one frame cannot be walking in the next. A valid transition requires at least several frames where the posture is classified as "standing-up". We present promising results and compare the classification rates of postures and gestures for the standard CSS and our new approach.
Many TV broadcasters and film archives are planning to make their
collections available on the Web. However, a major problem with large
film archives is the fact that it is difficult to search the content
visually. A video summary is a sequence of video clips extracted from
a longer video. Much shorter than the original, the summary preserves
its essential messages. Hence, video summaries may speed up the search
Videos that have full horizontal and vertical resolution will usually
not be accepted on the Web, since the bandwidth required to transfer
the video is generally very high. If the resolution of a video is
reduced in an intelligent way, its content can still be understood. We
introduce a new algorithm that reduces the resolution while preserving
as much of the semantics as possible.
In the MoCA (movie content analysis) project at the University of
Mannheim we developed the video summarization component and tested it
on a large collection of films. In this paper we discuss the
particular challenges which the reduction of the video length poses,
and report empirical results from the use of our summarization tool.