Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Reduction of Flicker in Four-Stroke Motion of Color Images:

Reduction of Flicker in Four-Stroke Motion of Color Images: When two sequential video frames extracted from a single video clip are followed by the negative of the two frames, a viewer often experiences a visual illusion whereby a scene in the frames continuously moves in a single direction (four-stroke apparent motion). To create a four-stroke apparent motion display, the image intensities of the whole of the second pair of images are reversed. However, this intensity reversal creates a strong impression of flicker that can be undesirable for comfortable viewing. This study reports that four-stroke apparent motion can be induced by only reversing the luminance intensities in those spatial areas which contain motion signals in high-pass filtered images. This use of only a partial reversal of image intensities greatly reduces the apparent flicker in the display while retaining motion perception. Keywords higher order motion, local motion, motion When a sequence of two video frames in which, say, a bright bar moves to the right are presented repeatedly, a viewer perceives an oscillation of the bar (Figure 1(a)). However, when the two video frames are followed by the same frames with the luminance polarity reversed (Figure 1(b)), the viewer perceives the motion of the bar as being in a consistent direction. This sort of illusory apparent motion is called four-stroke apparent motion (Anstis & Rogers, 1986; Mather, 1991). Four-stroke apparent motion displays stimulate the detector of the spatiotemporal image flow as shown in Figure 1(c). Because the movement of bright and dark bars, respectively, falls on the positive and negative lobes of the detector’s receptive field, a viewer perceives a consistent rightward motion. Those who want to add four-stroke motion to a natural image will normally try to reverse the intensities of complete image areas of all intensity channels. However, such an intensity reversal results in an intensive flicker, which is not always desirable for comfortable viewing. We aimed at reducing the apparent flicker by omitting those intensity reversals which were redundant. We focused on three redundant components: motion components at low spatial Corresponding author: Takahiro Kawabe, NTT Communication Science Laboratories, 3-1, Morinosatowakamiya Atsugi, Kanagawa 243-0198, Japan. Email: takkawabe@gmail.com Creative Commons CC-BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sage- pub.com/en-us/nam/open-access-at-sage). 2 i-Perception (a) (b) (c) Frame 1 Space Frame 2 Frame 3 Frame 4 Figure 1. (a) A sequence of video frames causing an apparent back-and-forth motion. (b) A sequence of video frames causing apparent unidirectional motion. (c) A schematic diagram to explain how a four-stroke apparent motion display stimulates the detectors of the spatiotemporal energy flow. frequency, chromatic components, and static components. In four-stroke motion, unlike the object’s edges, the uniform surface of a slowly moving object does not always contain effective image motion signals across two video frames. Thus, motion at low spatial frequencies, which are related to image information at the surface, may not contribute to four-stroke apparent motion, and so may be omitted. In addition, it is well known that chromatic components do not strongly contribute to motion perception (Kawabe, Fukiage, Sawayama, & Nishida, 2016; Ramachandran & Gregory, 1978), and hence the reversal of chromatic components may not be necessary for producing four-stroke apparent motion of color images. Finally, in the previous method, intensity reversals are applied even to static regions, which may not contribute to motion perception. Thus, we should apply intensity reversals only to those spatial regions that contain motion signals. Figure 2 shows the processing pipeline of our technique. Our technique aims at converting an original frame (Figure 2(a)) into an image with luminance intensity reversals in spatial areas with motion signals in a high-pass filtered image (Figure 2(b)). First, we obtain color and luminance channels of the original image by converting the RGB color space of the frame into, for example, CIE LAB (Figure 2(b)). The color channel image (Figure 2(c)) is kept intact. The image in the luminance channel is further decomposed into low-pass (Figure 2(d)) and high-pass (Figure 2(e) and (f)) filtered images. The cut-off frequency is a free parameter that users need to determine based on the magnitude of displacement in the sequence: Larger displacements require lower cut-off frequencies. Next, we determine motion and static areas on the basis of an optical flow calculation between the two frames (Figure 2(g) and (h)). Areas having a motion velocity that is higher or lower than 1 (pixels) are considered as static and motion areas, respectively. In Figure 2(e) and (f), static and motion areas are given values of 0 and 1, respectively. To apply intensity reversals only to motion areas, we multiply the intensity-reversed high-pass image (Figure 2(e)) by the motion areas and the unchanged intensity high-pass image (Figure 2(f)) by the static areas. By combining the color channel image (Figure 2(c)), the low-pass filtered image (Figure 2(d)), Time Kawabe and Nishida 3 (a) Original (b) Intensity reversal in (i) Intensity reversal motion areas of at all channels high-pass filtered image (c) Color channels (d) Low-pass filtered image (e) High-pass filtered (g) Motion areas image with intensity reversal (f) High-pass filtered image without (h) Static areas intensity reversal Figure 2. A schematic explanation of our method of creating an intensity-reversed video frame for four- stroke apparent motion. (a) Original, (b) intensity reversal in motion areas of high-pass filtered image, (c) color channels (d) low-pass filtered image, (e) high-pass filtered image with intensity reversal, (f) high-pass filtered image without intensity reversal, (g) motion areas, (h) static areas, and (i) intensity reversal at all channels. and the manipulated high-pass filtered images (Figure 2(e) and (f)), an image with luminance intensity reversal in the motion areas of the high-pass filtered image (Figure 2(b)) is obtained. The resulting image is more similar to the original image (Figure 2(a)) in comparison with an image that has undergone intensity reversals over the total image area and for all intensity channels (Figure 2(i)). Examples of the resultant four-stroke motion displays are shown in Videos 1 and 2, each of which shows a comparison with the original four-stroke motion displays. In an experiment, we presented, side by side, one of clips and the clip played backward on a Luminance channel 4 i-Perception CRT display and asked each of nine observers to answer the following questions by pressing assigned keys. For Video 1, the question was ‘‘which of waterfalls went downwards?’’ and for Video 2, the question was ‘‘which of the two soap bubbles went upwards?’’ Each of clips was tested 20 times. We tested four clips in total. The results showed that the proportion of observations identifying a correct clip was 0.948 on average with the standard deviation across the observers of 0.052. The same nine observers also compared the subjective magnitude of flicker between a video created using an original four-stroke motion and one created using our technique. All of the observers reported that the videos created with our technique had weaker flicker than those with the original four-stroke motion. By combining the various modifications, our technique provides a four-stroke apparent motion with reduced flicker without hampering motion perception. Our technique could, possibly, be of use to vision science researchers who want to clarify how color and luminance are integrated in motion processing. However, the determination of the cut-off frequency for the high-pass filter is left as an open issue for future study. In addition, our technique does not work with a long observation distance because motion at a relatively high spatial frequency bands is critical here. Even with these limitations, our technique provides a strong tool for visually representing continuous motion in a more comfortable and attractive way. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) received no financial support for the research, authorship, and/or publication of this article. Supplementary Material Supplementary material for this article is available online. References Anstis, S. M., & Rogers, B. J. (1986). Illusory continuous motion from oscillating positive-negative patterns: Implications for motion perception. Perception, 15, 627–640. doi: 10.1068/p150627 Kawabe, T., Fukiage, T., Sawayama, M., & Nishida, S. (2016). Deformation lamps: A projection technique to make static objects perceptually dynamic. ACM Transactions on Applied Perception, 13, 10. doi: 10.1145/2874358 Mather, G. (1991). First-order and second-order visual processes in the perception of motion and tilt. Vision Research, 31, 161–167. doi: 10.1016/0042-6989(91)90083-H Ramachandran, V. S., & Gregory, R. L. (1978). Does colour provide an input to human motion perception? Nature, 275, 55–56. doi: 10.1038/275055a0 Kawabe and Nishida 5 Author Biographies Takahiro Kawabe is a senior research scientist, Sensory Representation Group, Human Information Science Laboratories, NTT Communication Science Laboratories. He received a Doctor of Psychology from Kyushu University, Fukuoka, in 2005. In 2011, he joined NTT Communication Science Laboratories, where he studies applied aspects of human perception. He received the 2013 JPA Award for International Contributions to Psychology: Award for Distinguished Early and Middle Career Contributions from the Japanese Psychological Association. He is a member of the Vision Sciences Society and the Vision Society of Japan. Shin’ya Nishida is a senior distinguished scientist, Group Leader of Sensory Representation Group, Human Information Science Laboratories, NTT Communication Science Laboratories. He received a B.S., M.S., and Ph.D. in psychology from Kyoto University in 1985, 1987, and 1996. After a two-year stay at ATR Auditory and Visual Perception Laboratories as a research associate, he joined NTT in 1992. He is an expert in psychophysical research on human visual processing, in particular motion perception, cross-attribute/modality integration, time perception, and material perception. He is the current president of the Vision Society of Japan and an editorial board member of the Journal of Vision. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png i-Perception SAGE

Reduction of Flicker in Four-Stroke Motion of Color Images:

i-Perception , Volume 9 (1): 1 – Jan 11, 2018

Loading next page...
 
/lp/sage/reduction-of-flicker-in-four-stroke-motion-of-color-images-SomuyO0Y8I
Publisher
SAGE
Copyright
Copyright © 2022 by SAGE Publications Ltd unless otherwise noted. Manuscript content on this site is licensed under Creative Commons Licenses
ISSN
2041-6695
eISSN
2041-6695
DOI
10.1177/2041669517750400
Publisher site
See Article on Publisher Site

Abstract

When two sequential video frames extracted from a single video clip are followed by the negative of the two frames, a viewer often experiences a visual illusion whereby a scene in the frames continuously moves in a single direction (four-stroke apparent motion). To create a four-stroke apparent motion display, the image intensities of the whole of the second pair of images are reversed. However, this intensity reversal creates a strong impression of flicker that can be undesirable for comfortable viewing. This study reports that four-stroke apparent motion can be induced by only reversing the luminance intensities in those spatial areas which contain motion signals in high-pass filtered images. This use of only a partial reversal of image intensities greatly reduces the apparent flicker in the display while retaining motion perception. Keywords higher order motion, local motion, motion When a sequence of two video frames in which, say, a bright bar moves to the right are presented repeatedly, a viewer perceives an oscillation of the bar (Figure 1(a)). However, when the two video frames are followed by the same frames with the luminance polarity reversed (Figure 1(b)), the viewer perceives the motion of the bar as being in a consistent direction. This sort of illusory apparent motion is called four-stroke apparent motion (Anstis & Rogers, 1986; Mather, 1991). Four-stroke apparent motion displays stimulate the detector of the spatiotemporal image flow as shown in Figure 1(c). Because the movement of bright and dark bars, respectively, falls on the positive and negative lobes of the detector’s receptive field, a viewer perceives a consistent rightward motion. Those who want to add four-stroke motion to a natural image will normally try to reverse the intensities of complete image areas of all intensity channels. However, such an intensity reversal results in an intensive flicker, which is not always desirable for comfortable viewing. We aimed at reducing the apparent flicker by omitting those intensity reversals which were redundant. We focused on three redundant components: motion components at low spatial Corresponding author: Takahiro Kawabe, NTT Communication Science Laboratories, 3-1, Morinosatowakamiya Atsugi, Kanagawa 243-0198, Japan. Email: takkawabe@gmail.com Creative Commons CC-BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sage- pub.com/en-us/nam/open-access-at-sage). 2 i-Perception (a) (b) (c) Frame 1 Space Frame 2 Frame 3 Frame 4 Figure 1. (a) A sequence of video frames causing an apparent back-and-forth motion. (b) A sequence of video frames causing apparent unidirectional motion. (c) A schematic diagram to explain how a four-stroke apparent motion display stimulates the detectors of the spatiotemporal energy flow. frequency, chromatic components, and static components. In four-stroke motion, unlike the object’s edges, the uniform surface of a slowly moving object does not always contain effective image motion signals across two video frames. Thus, motion at low spatial frequencies, which are related to image information at the surface, may not contribute to four-stroke apparent motion, and so may be omitted. In addition, it is well known that chromatic components do not strongly contribute to motion perception (Kawabe, Fukiage, Sawayama, & Nishida, 2016; Ramachandran & Gregory, 1978), and hence the reversal of chromatic components may not be necessary for producing four-stroke apparent motion of color images. Finally, in the previous method, intensity reversals are applied even to static regions, which may not contribute to motion perception. Thus, we should apply intensity reversals only to those spatial regions that contain motion signals. Figure 2 shows the processing pipeline of our technique. Our technique aims at converting an original frame (Figure 2(a)) into an image with luminance intensity reversals in spatial areas with motion signals in a high-pass filtered image (Figure 2(b)). First, we obtain color and luminance channels of the original image by converting the RGB color space of the frame into, for example, CIE LAB (Figure 2(b)). The color channel image (Figure 2(c)) is kept intact. The image in the luminance channel is further decomposed into low-pass (Figure 2(d)) and high-pass (Figure 2(e) and (f)) filtered images. The cut-off frequency is a free parameter that users need to determine based on the magnitude of displacement in the sequence: Larger displacements require lower cut-off frequencies. Next, we determine motion and static areas on the basis of an optical flow calculation between the two frames (Figure 2(g) and (h)). Areas having a motion velocity that is higher or lower than 1 (pixels) are considered as static and motion areas, respectively. In Figure 2(e) and (f), static and motion areas are given values of 0 and 1, respectively. To apply intensity reversals only to motion areas, we multiply the intensity-reversed high-pass image (Figure 2(e)) by the motion areas and the unchanged intensity high-pass image (Figure 2(f)) by the static areas. By combining the color channel image (Figure 2(c)), the low-pass filtered image (Figure 2(d)), Time Kawabe and Nishida 3 (a) Original (b) Intensity reversal in (i) Intensity reversal motion areas of at all channels high-pass filtered image (c) Color channels (d) Low-pass filtered image (e) High-pass filtered (g) Motion areas image with intensity reversal (f) High-pass filtered image without (h) Static areas intensity reversal Figure 2. A schematic explanation of our method of creating an intensity-reversed video frame for four- stroke apparent motion. (a) Original, (b) intensity reversal in motion areas of high-pass filtered image, (c) color channels (d) low-pass filtered image, (e) high-pass filtered image with intensity reversal, (f) high-pass filtered image without intensity reversal, (g) motion areas, (h) static areas, and (i) intensity reversal at all channels. and the manipulated high-pass filtered images (Figure 2(e) and (f)), an image with luminance intensity reversal in the motion areas of the high-pass filtered image (Figure 2(b)) is obtained. The resulting image is more similar to the original image (Figure 2(a)) in comparison with an image that has undergone intensity reversals over the total image area and for all intensity channels (Figure 2(i)). Examples of the resultant four-stroke motion displays are shown in Videos 1 and 2, each of which shows a comparison with the original four-stroke motion displays. In an experiment, we presented, side by side, one of clips and the clip played backward on a Luminance channel 4 i-Perception CRT display and asked each of nine observers to answer the following questions by pressing assigned keys. For Video 1, the question was ‘‘which of waterfalls went downwards?’’ and for Video 2, the question was ‘‘which of the two soap bubbles went upwards?’’ Each of clips was tested 20 times. We tested four clips in total. The results showed that the proportion of observations identifying a correct clip was 0.948 on average with the standard deviation across the observers of 0.052. The same nine observers also compared the subjective magnitude of flicker between a video created using an original four-stroke motion and one created using our technique. All of the observers reported that the videos created with our technique had weaker flicker than those with the original four-stroke motion. By combining the various modifications, our technique provides a four-stroke apparent motion with reduced flicker without hampering motion perception. Our technique could, possibly, be of use to vision science researchers who want to clarify how color and luminance are integrated in motion processing. However, the determination of the cut-off frequency for the high-pass filter is left as an open issue for future study. In addition, our technique does not work with a long observation distance because motion at a relatively high spatial frequency bands is critical here. Even with these limitations, our technique provides a strong tool for visually representing continuous motion in a more comfortable and attractive way. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) received no financial support for the research, authorship, and/or publication of this article. Supplementary Material Supplementary material for this article is available online. References Anstis, S. M., & Rogers, B. J. (1986). Illusory continuous motion from oscillating positive-negative patterns: Implications for motion perception. Perception, 15, 627–640. doi: 10.1068/p150627 Kawabe, T., Fukiage, T., Sawayama, M., & Nishida, S. (2016). Deformation lamps: A projection technique to make static objects perceptually dynamic. ACM Transactions on Applied Perception, 13, 10. doi: 10.1145/2874358 Mather, G. (1991). First-order and second-order visual processes in the perception of motion and tilt. Vision Research, 31, 161–167. doi: 10.1016/0042-6989(91)90083-H Ramachandran, V. S., & Gregory, R. L. (1978). Does colour provide an input to human motion perception? Nature, 275, 55–56. doi: 10.1038/275055a0 Kawabe and Nishida 5 Author Biographies Takahiro Kawabe is a senior research scientist, Sensory Representation Group, Human Information Science Laboratories, NTT Communication Science Laboratories. He received a Doctor of Psychology from Kyushu University, Fukuoka, in 2005. In 2011, he joined NTT Communication Science Laboratories, where he studies applied aspects of human perception. He received the 2013 JPA Award for International Contributions to Psychology: Award for Distinguished Early and Middle Career Contributions from the Japanese Psychological Association. He is a member of the Vision Sciences Society and the Vision Society of Japan. Shin’ya Nishida is a senior distinguished scientist, Group Leader of Sensory Representation Group, Human Information Science Laboratories, NTT Communication Science Laboratories. He received a B.S., M.S., and Ph.D. in psychology from Kyoto University in 1985, 1987, and 1996. After a two-year stay at ATR Auditory and Visual Perception Laboratories as a research associate, he joined NTT in 1992. He is an expert in psychophysical research on human visual processing, in particular motion perception, cross-attribute/modality integration, time perception, and material perception. He is the current president of the Vision Society of Japan and an editorial board member of the Journal of Vision.

Journal

i-PerceptionSAGE

Published: Jan 11, 2018

Keywords: higher order motion; local motion; motion

References