Part II - Motion Trails (using Edge-based Sharpening)

Explanation

Using the edge detection method previously, edges are detected on frames 1, 4 and 7 out of the collection of all frames, which is defaulted to 12. Each of these selected frames is then added onto the original video by adding up the RGB values. Since the edge detection algorithm only provides the edges and nothing else, the RGB components can be added directly without the need to edit alpha. The result is expected to show some sort of trails of the edges. This is achieved as seen below, reflected by the blue lines.

I'm afraid you'll need to click on the image to have a clearer view.

Question

1. If you used a video of people moving as a crowd, do you think it would be useful?

Yes, it will be useful. However, it may be computationally expensive as a lot of edges need to be calculated. Apart from that, the trails would be a very nice effect to have, visually.

Reflection 

Originally a headache to comprehend, the tutorials shed some light on the solution that I needed. Motion trails are basically just trails or outlines being displayed in a delayed manner. What I did was, I took the Laplacian algorithm as a function that returns the edges for a single frame. After gathering 12 frames, I process the first, fourth and seventh frame to get the edges that will form a trail. By now, you might've noticed that the video played is already 12 frames ahead. So what I thought I'd do is that, I'll just add these processed frames (with edges) onto the front canvas and repeat this until the video ends. Turns out I was right, and the above shows the result of my thought process.

Part II - Slow-response Threshold

Explanation

Slow-response Threshold is similar to doing a threshold on an image, except that this one is done using a collection of frames (images). Each frame is stored in a stack, where their RGB values are added into 3 different histograms, one for each color. From there, the mean is obtained for each individual RGB component, and is used as the threshold. When the next frame arrives, the count from the oldest frame in the stack is removed from the histogram before adding the new counts from the latest frame.

 The result is as below:

Uhh...I think you won't be seeing the effect much because...

yeah well, the above isn't a video. Sorry about that.

Question

1. Does the slowly modifying threshold improve visibility of the video content?

Not really, in my opinion. Since there are differences in colors, certain parts of the video are more difficult to see. Maybe it will work better on videos with simple content such as having only 1 background color and 1 simple object (a shape or something similar) moving around. However, I noticed that the colors here are quite vivid.

Reflection 

I initially had no idea if my output was right. So during the presentation I kind of asked Mr. Rimi and he gave me yet another answer which was so obvious, yet it crossed not my mind. Slow-response thresholding is just a form of thresholding done in a delayed manner with reference to the current frame. If I were to use a video which changes scene from one to another, the effect would've been much more evident. I think I need to somewhat train myself to think more. Talk about slow response, huh..?

Part II - Edge-Based Sharpening

Explanation

Edge-based Sharpening applies Laplacian filtering, hence using the convolution method from Part I - Edge Detector in the first assignment. Instead of applying it on one image, the filter is applied onto every frame.

As for calculating edge based on frame difference, the previous frame data is stored when processing the next frame. By having data for both frames, the differences are obtained by subtracting the pixel data of the current frame from the previous frame. This difference is then placed onto the front canvas to be displayed.

Below is a sample screenshot:

Reflection

Right, the convolution process again. The algorithm has already mind-boggled me in Part I, and it came back to haunt me here again. But rather than shying away, I think I learnt new things here too. Laplacian filter, as we all know, is only a matrix of values. Change these values and you might get another effect including:-

• sharpened images
• blurred images
• Sobel filtering (horizontal/vertical)

So since we can do all these using a single algorithm with different matrix values, I wonder...would I get the same effect if I used other different algorithms? If the effects are different, how do we determine which one is best? How do we determine which is the one we really want? And how would I know which I should use? More and more interesting questions, but let's keep that for another time.

Part II - Color Threshold

Explanation

This requirement is similar to the Color Threshold in Part I. It sets the threshold of 50% for all the RGB color and depending on that threshold, RGB values which are higher are set to 255 while the lower ones are set to 0. This increases/decreases the color on that particular frame.

The outcome is expected to be a video in which the RGB colors can be altered depending on the threshold. Below is a sample screenshot:

Question

1. What is the effect of this on running videos?

The effect is that each frame displayed inherits from the manipulated data in which the RGB values are altered. Since a video is a collection of images (frames) and each frame is altered individually, the outcome will be a running video with changeable colors.

Reflection
On the surface level, this is essentially the same as Part I - Color Threshold. I like the structure of this assignment where Part I trains you with the basics while Part II takes that to the next level. The learning process here gives me the chance to play around with videos by handling frame by frame. Now, with this knowledge at hand, I should theoretically be able to play around with videos the way I want things to be- not just to answer this assignment question.

I think this is what education should be like- to be granted freedom and nurtured in such a way that we as students are more inclined to explore the many different things by our own ways and methods of thinking.

Oops, I think I went off-topic. Moving on . . .

Part II - Zebra Stripes for Videos

Explanation

This requirement is basically applying Zebra Stripes for Images into each frame of the video. Three optional sliders are also given to control the threshold value for each of the RGB component. Then, if the gray value falls within the zebra boundaries, that pixel’s component is set to the value determined by the slider.

The outcome is expected to be the same as Zebra Stripes for Images , except with the addition of colors to replace black. Below is a sample screenshot:

Hooray, green color!

Question

1. What would be the use of such a filter for videos?

The use for this filter is similar to the previous requirement, except that it is applied on videos here. Additionally, colors can be used instead of using black color as done previously.

2. What kind of videos would best show the effect?

Videos with gradient! Yes, of course, the answer I wrote in the report is misleading. So after Mr. Rimi went through the trouble of making the gradient image I spoke about earlier into a video, I'd say such examples are the best to display the effect of zebra stripes. After all, a video is actually a collection of still frames aka images.

Reflection
I was thinking quite differently before Mr. Rimi showed us the gradient image & video. I was thinking more along the lines of... "if the background (like the wall in the image above) falls into the zebra category while the object (in this case, that handsome guy in the image) in the foreground doesn't, it'd show a difference". But when I think about it again, that'd only work if the objects in the foreground is simple in color....right?

Part II - Zebra Stripes for Images

Explanation

This requirement is similar to the Gray Threshold. Instead of taking a threshold value, the gray value for each pixel is compared with a zebra filter. Gray values ranging between:

0-31, 64-95, 128-159, 192-223

 are set to black while the rest retain their gray pixels. The expected outcome is a blend between black and gray color. This is shown by the output below:

Question

1. What would be the use of such a filter for images?

The image filter makes colors with similar grayscale values to be recognized (by turning them into black, or setting them to gray). This allows the detection of color brightness which can then be filtered based on the zebra strip width to display only gray values for a certain range.

2. What does the effect look like?

It looks like a messy grayscale image with a bit of distortion in terms of edges. At certain parts (the hair) edges seem to be less obvious although the wooden panels in the background seem to be distinctively clear. Smooth surfaces also seem to be less implied here, but that is dependent on color.

Reflection
Actually, I don't know how to describe the image. Earlier, I demonstrated this assignment to Mr. Rimi who is the lecturer in charge. He created a gradient image (white slowly turning to black) and we used that image to experiment these zebra stripes. The effect was much more clearer there as I could see stripes alternating between the original pixels and the stripe-altered regions.

I went like 'oooohhhhhhhhhhhhhhh!', literally.

Part I - Edge Detector

Explanation

In this requirement, two canvases were used where one served as a back-canvas to draw the image and perform calculations, and the other canvas is used to display the final output. Using the back-canvas, the convolution algorithm is applied using the Laplacian filter to detect edges in the image. The filter is basically an array of 9 elements, but the algorithm reads it as a 3x3 neighborhood operation.

When applied, the algorithm traverses through each pixel in the image. For each pixel, the neighbor pixels are calculated and then the filter is applied according to the given values. The alpha value is always set to 255 so that the output will be visible. At the end, the output is drawn onto the front canvas.

The expected output is a semi black-and-white image with edges making out the shape of the ‘objects’ present in the image. Below is the result:

Reflection

The challenge in this requirement is huge to me. The convolution algorithm is difficult to understand with examples in the tutorial having little to no notes attached. It was difficult trying to search on Google, but eventually I stumbled upon html5rocks and learned from there. As soon as I saw that website, I was in awe at the power of HTML 5 and how much technology has developed over the years. A few years back, I still remember doing web programming as a subject. Ah, the nostalgia...

Part I - Color Detector

Explanation

For this requirement, two threshold values are used to determine each pixel’s color values. Using Color Threshold’s knowledge of accessing and altering RGB components, each pixel’s component is compared against two respective threshold values. If the value is lower than the lower boundary or higher than the upper boundary, that component’s value is set to black. If the value is between the lower and upper boundary, the value is set to 255 which increases that pixel’s component color.

Above is an image where the red threshold values are altered. Other colors are not present because their threshold values have no middle region (128, 128). The expected results depend on threshold values and the image itself (RGB in each pixel).

Question

1.       Describe how you could use this for face detection.

By using two different suitable thresholds, we can control the pixels to be displayed if they are within the threshold boundary. The pixels outside the boundary can then be altered to be ‘edited’ out, such as setting them to black. This way, we can get the image of the face alone, assuming correct thresholds were used.

2.       Would this method be light-dependent? In other words, will it work on the beach on a sunny day as well as indoors, or do you have to change settings?

I think it is light-dependent. Since this method relies on color detection, it is likely to mistake some other parts of the picture to be the face especially when the person’s face and the background are similar in color. Additionally, if a source of colored light is shone upon the part or most of person’s face, it will be even more difficult to detect the skin tone, hence not resulting in accurate face detection.

Reflection

An unexpected difficulty I faced while doing this is using the .value to compare between each slider’s values. There are parts where I can definitely tell that A.value is bigger than B.value, but JavaScript took it the other way around. (I confirmed this by using ‘alert’ to display out the values). I figured it might be because of some string-related issues, so I fixed it using Number(X.value) which put my half-an-hour-of-stressful-troubleshooting to an end.

After doing the histograms and things like that, I feel like I am getting better at using histograms. Forgive me for being happy here; it's a very small step but it feels really good to know I've improved!

Part I - Color Threshold

Explanation

Similar to Gray Threshold previously, the same concept of threshold is applied. Instead of using gray values, the RGB components are compared directly with the threshold, then altered accordingly. Setting a component value to 0 will result in that particular color’s absence, while setting it to 255 will increase the intensity of that component’s color.

Optimal threshold value is obtained by first constructing a histogram for each of the RGB component. The mean is then calculated separately to get the optimal value to be used as a threshold. The expected outcome is an image clear enough to be identified as similar with the original image, only with colors adjusted. Output is as below:

Question

1. Explain how many different colours are possible using this method of ‘binary’ thresholding?

Thresholding is a method of image segmentation and as such, it can be used to create binary images. Binary images only have two possible values for each pixel, hence the different combinations can be calculated as 2^x where x is the number of pixels.

Reflection 

The challenge posed in this requirement is how to calculate the mean given that the histogram is only constructed using a 2D array. For each histogram index, it is multiplied with the number of counts and then summed up. That value is then divided by the sum of all indexes to obtain the mean value. In the previous requirement, we dealt with grayscale. Here, we alter the RGB components independently. Lena looks much more fanciful now that the colors are brighter.

Part I - Gray Threshold

Explanation
So, to perform a threshold, I used the gray value obtained by using either:

1. (R + G + B) / 3
2. (0.3*R) + (0.59*G) + (0.11*B)

The gray value is then compared with the threshold, where if the gray value is above the threshold, the entire RGB value for that particular pixel is set to white. Otherwise, they are set to black. A slider is also provided to control the threshold value between 0-255. The initial threshold is set to 50%, which is roughly 128. A black & white output is expected and a sample screenshot is provided below.

Question

1. Would the threshold calculated by one method be equivalent to that calculated by the second method? Why / why not?

No. The first method, (R + G + B) / 3 simply returns the average values among the three colors. The other method, (0.3*R) + (0.59*G) + (0.11*B) returns the weighted average, since each color carries a different weight with regards to perceptions made by the human eye.

Reflection 

I initially thought grayscale thresholding would result in a grayscale image, but it seems that I was wrong. At least according to the requirement's intructions. Doing this was fairly easy, but unknowingly, this became the base for all other requirements. In other words, this is where I learned and understood about pixel and RGB components access.

(The Opposite of) Beginning

Hello, welcome to this simple blog. You must have already read some of the posts above, because this will technically be the last post you should be seeing. So I'll make this post short just for your convenience.

Thank you very much for stopping by and reading this blog. I hope you had fun. Have a nice day ahead!

Until then, farewell.