A11 - Camera Calibration

July 31, 2008



In this activity, we try to align the world coordinate system, which pertains to the real three-dimensional world of an object, with the two-dimensional image coordinate system captured by a camera.


We get the transformation matrix that would translate all world coordinates to the image coordinates. This transformation matrix depends on the angle the optic axis makes with respect to the object to be captured and on the intrinsic properties of the camera which are mainly affected by its focus. Denoted by A, it can be found by first realizing that













(Equation taken from Dr. Maricor Soriano's lecture notes.)


where xo, yo, zo are real world coordinates and yi, zi are image coordinates. Thus, A is given by

A = [ (inv(Q'Q)) Q'] d

where Q is the first matrix given in equation 1 and d is the column vector at the the right hand side of the equation. When there are N sample points, Q is a 2N x 11 matrix while d has a dimension of 2N x 1.

After we have determined A, we tried to predict the image coordinates of world points that were not previously sampled. The whole process is summarized by figure 1. A celfone camera was used to capture a checkerboard with two arms positioned perpendicular to each other. Each cell in the checkerboard is 1 inch x 1 inch. Twenty-five random points and their 2D projection on the image plane were used to determine the transformation matrix. Afterwards, the position in the image of five other points were interpolated using the transformation matrix. With the z plane in the real world chosen to be vertical, the left arm was designated to be the xz plane. The right arm is then the yz plane.

We could see that the predictions are accurate since some points even blocked the labeling numbers. Prediction #2, which missed the theoretical position, is not too far from where it should be.
























Self-grade: 10/10
Credits: Benj for some clarifications on the later part of the problem


//Camera Calibration
//1 Image space
d = [135 138;230 165;177 188;229 259;178 91;279 141;261 216;110 213;244 89;159 260;179 233;135 89;315 137;36 137;63 266;64 58;315 311;63 322;317 21;39 32;38 242;180 47;336 227;88 189;297 53];
//2 Object space
s = [3.5 0 1.5;0 1.5 0.5;1.5 0 -0.5;0 1.5 -3.5;1.5 0 3.5;0 4.5 1.5;0 3.5 -1.5;4.5 0 -1.5;0 2.5 3.5;2.5 0 -3.5;1.5 0 -2.5;3.5 0 3.5;0 6.5 1.5;7.5 0 1.5;6.5 0 -3.5;6.5 0 4.5;0 6.5 -4.5;6.5 0 -5.5;0 6.5 5.5;7.5 0 5.5;7.5 0 -2.5;1.5 0 5.5;0 7.5 -1.5;5.5 0 -0.5;0 5.5 4.5];
//3 Finding the transformation matrix a
Q = zeros(50, 11);
i = 1;
j = 2*i-1;
while i<>
Q(j,1:3) = s(i,1:3);
Q(j,4) = 1;
Q(j,5:8) = 0;
Q(j,9:11) = -d(i,1)*s(i,1:3);
Q(j+1,1:4) = 0;
Q(j+1,5:7) = s(i,1:3);
Q(j+1,8) = 1;
Q(j+1,9:11) = -d(i,2)*s(i,1:3);
i = i+1;
j = 2*i-1;
end;
d = matrix(d',50,1);
a = inv(Q'*Q)*Q'*d;
//Predicting the image coordinates of unsampled world points
test = [3 0 0 1;0 5 -3 1;0 3 2 1;1 0 2 1;3 0 -2 1];
newd = a*test';
newd = matrix(newd,5,3);
newd = newd';
[m,k] = size(M);
overlay = zeros(m,k);
for i = 1:size(newd,2);
overlay(round(newd(2,i)),round(newd(1,i))) = 1;
end
M = imread('D:\ap186\july29_2008\cropped.jpg');
M = im2gray(M);
subplot(121)
imshow(M,[]);
subplot(122)
imshow(overlay);

A11 - Preprocessing Handwritten Text

First, we get chose a portion of the grayscaled image. Then we, removed the vertical lines by knocking-out the frequencies in the Fourier transform of the image that correspond to vertical lines. These are those frequencies that lie along the vertical axis of the shifted FFT. So the mask used was a black vertical strip with a hole in the middle so as not to remove the DC component of the image.

Next, I binarized the image. I used thresholds = 80, 90 and 100 intensity values and found out that threshold = 80 is the best. So I, tried applying the operations closing of holes and opening of holes for the image and also opening the holes for holes-closed image. I think that the image became cleaner after performing closing of holes only. Next, I thinned the characters/regions into one pixel. The final image is shown in figure 6. While not all letters are recognizable, the letters U,P and S have been, in my opinion, enhanced.

Enjoy mag-label ng continuous regions.

Code:

//1 Calls the image and converts it to grayscale

M = imread('D:\ap186\july20_2008\toEnhance3.jpg');

M = im2gray(M);

M = M-min(min(M));

M = 255*M/max(max(M));

//3 Shows the FFT of the best binarized image,K2

FM = fft2(M);

FMshift = fftshift(abs(FM));

//4 Creation of a mask to filter the horizontal and vertical lines in the image

[m,n] = size(M);mask = ones(m,n);

mask(:,n/2-5:n/2+5) = 0;mask(m/2-3:m/2+3,n/2-3:n/2+3) = 1;

//5 Shows the image FFT and the mask to be used

subplot(121)

imshow(log(FMshift),[]);

subplot(122)

imshow(mask);

//6 Filtering of the lines; the new image is called H

FHM = FM.*fftshift(mask);

H = abs(ifft(FHM));

//7 Shows the original image, the filtered image, and the latter's FFT

subplot(131)

imshow(M,[]);

subplot(132)

imshow(H,[]);

subplot(133);

imshow(fftshift(abs(FHM)),[]);
H = H-min(min(H));

H = 255*H/max(max(H));

subplot(221)

histplot([0:255],H);

subplot(222)

L1 = im2bw(H,90/255);

L1 = 1-abs(L1);

imshow(L1);

subplot(223);

L2 = im2bw(H,100/255);

L2 = 1-abs(L2);

imshow(L2);

subplot(224);

L3 = im2bw(H,110/255);

L3 = 1-abs(L3);

imshow(L3);

//Close or Open holes for L1
L = ones(2,2);

zero = zeros(2,2);

negL = abs(zero-L);

D = dilate(L1,negL);

Close = erode(D,negL);
E = erode(Close,L);

Open = dilate(E,L);

subplot(221)

imshow(L2);

subplot(222)

imshow(Close);

subplot(223)

imshow(Open);

subplot(224)

thn = thin(Close);

imshow(thn);
lbl = bwlabel(thn);

imshow(lbl,jetcolormap(max(max(lbl))));

Grade: 8/10

processed image is not so readable

A10 - Morphological Operations

We tried to get the best estimate of the area of a punched paper by statistical analysis of a set of punched papers. We cut the image into six sub-images such as the one in figure 1 and converted them into grayscale. We looked at the histogram of their intensity values and determined that the threshold value should be set at intensity = 200.

We then performed the Morphological operations of opening and closing of holes to see what would happen. Opening of holes means to dilate with a structuring element L the image eroded with L. Closing of holes means to erode with the reflection of L the image dilated by the reflection of L. It seems that closing of holes had a better effect of producing a cleaner binary image. So I chose to continue processing this.

As shown in figure 3, the closed regions were labeled with a specific number. The area of each labeled region is determined by pixel counting.




























The histogram of the above sample is merged with the histograms of other subimages. The result is the histogram shown above. It seemed that the histogram needed no further noise eradication. The best estimate for the area was 549 px.

To check, we get the number of included pixels in a binarized image of a single punched paper. It was 519. The deviation of the best estimate from this value was around 6%.














Code:
for i = 1:7;
//1 Call image, transform it to black-and-white
M = imread(strcat(string(i)+'.jpg'));
M = im2gray(M);M = M*255;
//subplot(121)
//imshow(M,[]);
//subplot(122)
//histplot([0:255],M);
threshold = 200;
K = im2bw(M,threshold/255);
//2 Close-openL = ones(5,5);
zero = zeros(5,5);
negL = abs(zero-L);
E = erode(K,L);
D = dilate(K,negL);
Open = dilate(E,L);
Close = erode(D,negL);
subplot(131)
imshow(K);
subplot(132)
imshow(Close);
subplot(133)
imshow(Open);
//3 Label Opened image because it looks good
lbl = bwlabel(Open);
N = max(max(lbl));
//imshow(lbl,jetcolormap(N));
//4 Count area of each similarly labeled region
Area = zeros(7,N);
for j = 1:N; [x,y] = find(lbl == i);
Area(i,j) = length(x);
end
end
histplot([1:max(max(Area))],Area);
mn = mean(mean(Area))
std = stdev(Area)
//5 Eradicate noiseupperbound = mn + std + 1;
lowerbound = mn - std - 1;
Area = Area.*(Arealowerbound);
histplot([1:max(area)],Area);

Credits:

Lei Uy and Beth Prieto for helpful discussions, Benj Palmares for help in histogramming.

Self-evaluation: 9/10

I fell happy when labelling the regions because the resulting image is colorful

A9 - Binary Operations

July 17, 2008

The goal of this activity is to perform area estimation of cells represented by punched paper.

In figure 1, part of the entire image is binarized using a threshold pixel value that is equal to 200 where the valley in the histogram occurs.
















Next, we performed higher order morphological operations on the binarized image: opening of holes and closing of holes. We let M be the original image, K the binary image and L the structuring element. The two operations are implemented in SciLab as

M = imread('C:\Documents and Settings\AP186user20\Desktop\Circles1.jpg');
M = M - 1;
histplot([0:255,M);
threshold = 200;
K = im2bw(M,threshold/255);
L = ones(5,5);
zero = zeros(5,5);
negL = abs(zero-L);
E = erode(K,L);
D = dilate(K,negL);
Open = dilate(E,L);
Close = erode(D,negL);





















Reference:
[1] http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip-Morpholo.html#Heading98

A8 Morphological Operations

July 15, 2008


In this activity, we tried to investigate the effects of dilation and erosion of an image by a structuring element. We predicted first what would happen and then simulated them in SciLab.
The structuring elements used were:

• 4x4 ones
• 2x4 ones
• 4x2 ones
• cross, 1pixel thick, 5 pixels long

We dilated and eroded images of a square, a triangle, a hollow square (60 x 60, 4px thick), a circle and a cross.

So far, most of my predictions were wrong. I was fairly good in guessing for the square, triangle and hollow square images and slightly for the cross. I was terrible with the circle.

I also realized that the image size I should should be larger than the shape of the actual square or triangle, etc. This is because if the input image Img is m x n, the dilated image is also m x n. Some parts were cut in the dilated image. But since in erosion the result is smaller than the image, the effect of this operation was more imminent.

Remarks:

Beth Prieto, Ayeen Laganapan, Rica Mercado for helping me grapple with concepts.

Julie Dado for assisting me in publishing a video.

Grade: 9/10

A. Square

B. Triangle

C. Hollow square

D. Circle

E. Cross

A7 - Enhancement in the Frequency Domain

July 10, 2008


A. Anamorphic Property of the Fourier Transform

We tried to get the Fourier transform of a sinusoidal object similar to a corrugated roof. The first sinusoid has a frequency of 2. Its Fourier transform is shown as two dots above and below the center of the FFT and symmetric about the origin. We explored what would happen if we increase the frequency to 4. The two spots move further away from each other. We rotated the sinusoid. The FFT was rotated in the opposite direction. Next, we superpose a horizontal and a vertical sinusoid, both with frequency = 2, by simply adding them. The resulting FFT is the superposition of the FFTs of each sinusoid, since we already know that the FFT of the vertical sinusoid is the same as that of the horizontal sinusoid only that it is rotated 90 degrees.





















































B. Fingerprints: Ridge Enhancement

Here, we try to enhance the picture of a fingerprint in such a way as to make the ridges more visible. We created an aperture shown below which we multiplied by the Fourier transform of the fingerprint image.











We made sure that the position of the mask corresponds to that of the secondary bright spots in the fingerprint image's FFT.

Code:
mask = imread('square6.bmp');
finger = imread('fingerprint2.jpg');
maskgray = im2gray(mask);fingergray = im2gray(finger);
FFT_finger_mask = fft2(fingergray).*fftshift(maskgray);
Image_filtered = abs(ifft(fftshift(FFT_finger_mask)));
Image_filtered = Image_filtered-mean(mean(Image_filtered));
scf,imshow(Image_filtered,[]);
scf, imshow(fingergray,[]);
scf,imshow(log(fftshift(abs(fft2(fingergray)))),[]);
scf,imshow(log(fftshift(abs(FFT_finger_mask))),[]);



C. Lunar Landing Scanned Pictures: Line Removal

Here, we try to remove the vertical lines regarded as artifacts in the image. The original photo is taken from http://www.lpi.usra.edu/lunar/missions/apollo/apollo_11/images. We get the FFT transform of the image. If it fringe-free, we expect that it should only possess a DC component or a zero-order frequency indicated by a very bright central spot. However, if this is not the case, we expect the spots that are less bright than the DC would appear. For the original image with vertical lines, the FFT had horizontal slits to the left and right of the DC component. We therefore, make a mask that would cover these slits and multiply them to the FFT of the original image. We see in the final image, that the vertical lines reduced in visibility.

Code:

M = imread('hi_res_vertical_lg.gif');

K = imread('slit2.bmp');

K = im2gray(K);

FFT_M = fft2(M);

FFT_MK = FFT_M.*fftshift(K);

imshow(log(fftshift(abs(FFT_MK))),[]);

MK = (ifft(FFT_MK));

MKimage = abs(MKimage)/max(max(abs(MKimage)));

scf,imshow(abs(MK),[]);

scf,imshow(log(fftshift(abs(FFT_M))),[]);

scf,imshow(M,[]);

SELF-EVALUATION: 10/10.

Fun activity, practical but tedious.

Credits:

Ma'am Jing, Andrew Banas, Gerold Pedemonte and Beth Prieto for answering some of my questions.

Fourier Transform Model of Image Formation

July 8, 2008

In this activity, we try to get the Fourier transform of an image.

A. Familiarization

The images are created in Paint. Programs are run in Scilab. First, we investigate a circle such as the one below. The input image size is 128 x 128.


Performing the built-in Fast Fourier transform (FFT) algorithm in Scilab on it, we get quadrants of concentric circles positioned in the corner of the FFT image.



We shift the FFT so that it is symmetric about the origin.
We obtain the Airy disk.





Next, we render the Airy disk in Scilab's hotcolormap, which is composed of shades of red and yellow.
B. Simulation of an Imaging Device

Now, suppose we have a camera which has a lens aperture. Upon passing through the aperture, the signals from this object are distorted or convoluted. Convolution of two functions f and g, representing the object and the aperture respectively, is equivalent to multiplying element-per-element their Fourier transforms, and getting the Fourier transform of their product. The resulting is what we see when the image forms at the back of the lens.

For example, we have this object below.


Its Fourier transform is,




The aperture is already in Fourier space, so we multiply it with VIP's FFT. Then, we get the FFT again. The result is a blurred mirror reflection, with visible fringes, of the object.










When we replace the lens with a larger one, the object is better resolved. Meanwhile, shifting to a smaller lens distorts the image more. This must be because the zero-order of the Airy disk broadens as the aperture gets smaller, thus causing more distortion.



































C. Template Matching Using Correlation


We also tried the application of the Fourier transform in Template Matching.
We have on the left side the input image, and below it the pattern we want to trace. Using the following code, to track down A, an inverted and blurred image of the input is retrieved. Notice the locations of the bright spots. They correspond to the positions of A in the input.




m = imread('C:\Documents and Settings\instru\Desktop\ap186(2)\fig11.bmp');
k = imread('C:\Documents and Settings\instru\Desktop\ap186(2)\fig12.bmp');
mgray = im2gray(m);
kgray = im2gray(k);
M = fft2(mgray);



K = fft2(kgray);
MK = M.*conj(K);
invMK2 = fft2(MK);

invMK2 = fftshift(invMK2);
imshow(abs(invMK2),[]);












D. Edge Detection

Let's go back to VIP. Now our aim is to detect the horizontal (top image) and vertical edges (bottom image).



m = imread('C:\Documents and Settings\instru\Desktop\ap186(2)\fig5.bmp');
k = [-1 2 -1; -1 2 -1; -1 2 -1];
mgray = im2gray(m);
l = imcorrcoef(mgray,k);
imshow(abs(l),[]);




















SELF -EVALUATION: 10/10.
FFT is amazing, especially in edge detection.



Reference:
[1] Dr. Soriano's lecture notes