|
The recent decade has seen tremendous growth in the machine vision applications as they become the new driving forces in precision engineering. Many of these applications highly benefit from the improvements of camera calibration techniques, which primarily fall into two categories: photogrammetry calibration1 and self-calibration.2 Compared with the latter which does not use any specific calibration target, the former usually provides higher accuracy by using a precisely-constructed calibration target and is therefore the interest of this letter. A representative photogrammetry approach is Zhang's method, which uses a planar pattern as the calibration target and is well known for its flexibility, robustness, and low cost.1 This method and its extensions implemented in the OpenCV library are considered as the conventional technique in this letter. The literature in the relevant field addresses two major sources of error that affect the camera calibration results. The first one is the imperfection of the calibration target. Since the assumptions made for the conventional camera calibration are based on a perfect planar target with ideal patterns, the imprecision of the calibration target may lead to inaccurate results.3 The second problem is the uncertainty in locating the control points directly from the geometries of the calibration patterns in the captured raw images which suffers from lens distortion as well as perspective distortion.4 This letter proposes an advanced geometric camera calibration approach to overcome the aforementioned two problems without loss of the original advantages of the conventional technique. The proposed technique uses a sophisticated lens distortion model that takes the radial, tangential, and prism distortion into account, and achieves a precise localization of the control points with a novel refinement process using a frontal image concept and an advanced digital image correlation (DIC) scheme; in addition, the imperfection of the calibration target can be compensated. For the ideal pinhole model, assuming that the world coordinate is placed on the calibration target with its surface as the XY plane, the relation between the three-dimensional world coordinate of a calibration target point [TeX:] \documentclass[12pt]{minimal}\begin{document}${\bm M} = [X_w, Y_w, Z_w]^T$\end{document} and its corresponding location [TeX:] \documentclass[12pt]{minimal}\begin{document}${\bm m} = [u,v]^T$\end{document} in the image plane can be expressed as: Eq. 1[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{equation} s \left\lbrace \begin{array}{c}\boldsymbol m\\[5pt] 1\\ \end{array}\right\rbrace = \boldsymbol A\left[ \begin{array}{cr}\boldsymbol R &\boldsymbol T \end{array} \right] \left\lbrace \begin{array}{c}\boldsymbol M\\[5pt] 1\\ \end{array} \right\rbrace , \ \ \boldsymbol A = \left[ \begin{array}{c@{\quad}c@{\quad}c}\alpha & \gamma & u_0 \\[5pt] 0 & \beta & v_0 \\[5pt] 0 &0 & 1 \\ \end{array} \right], \end{equation} \end{document}Eq. 2[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{eqnarray} u^{\prime } &=&(1+a_0r^2+a_1r^4+a_2r^6)u+s_0r^2 \nonumber \\ & &+\,(p_0+p_2r^2)(r^2+2u^2) ,\nonumber \\ v^{\prime } &=&(1+a_0r^2+a_1r^4+a_2r^6)v+s_1r^2\\ & &+\,(p_1+p_3r^2)(r^2+2v^2) ,\nonumber \\ r^2 &=& u^2+v^2,\nonumber \end{eqnarray} \end{document}Given the locations of the control points in the world coordinate as [TeX:] \documentclass[12pt]{minimal}\begin{document}${\bm M_{\bm i}}$\end{document} and in the image plane as [TeX:] \documentclass[12pt]{minimal}\begin{document}${\bm m_{\bm {ij}}}$\end{document} , the calibration process involves a nonlinear optimization with the cost function defined as: Eq. 3[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{equation} S = \sum _{i=1}^k \sum _{j=1}^l ||{\bm m_{\bm {ij}}}-P({\bm A},{\bm \varphi },{\bm r},{\bm T},{\bm M_{\bm i}})||^2, \end{equation} \end{document}The method described above relies on the intrinsic assumption that the control points on the calibration target are perfectly positioned. In reality, printing imprecision during calibration pattern fabrication is inevitable, especially for low-cost hardware systems. To deal with this issue, the proposed technique allows the world coordinates of the control points to be unknown, and they will be accurately determined together with the camera parameters. To ensure the uniqueness of the calibration target during optimization, a geometric constraint on three noncollinear control points, named markers, is applied.6 The constraint enforces planarity of the markers by setting their Z world coordinates to zero, and requires the distance between any two of the three markers to be accurately measured to solve for the scale factor. The markers, normally made at the corners of the calibration panel, also help determine the orientation of the panel. An accurate ellipse fitting technique can be employed to detect the centers of the calibration circles.7 However, as the control points detected in the raw images suffer from lens and perspective distortion, their true locations cannot be accurately determined, and this will further lead to calibration errors. To cope with this issue, a two-step refinement with the frontal image concept, which is free from lens and perspective distortion, is employed. The refinement is conducted after the camera parameters are coarsely determined using the method described above. The raw images are successively undistorted by applying Eq. 2 and then reversely projected to the world coordinate system through using Eq. 1. The frontal image is generated from a direct scaling of the world coordinates. It is noteworthy that while the conventional subpixel edge and circle fitting method can be used to detect the control points in the frontal images, the method is not recommended because the gradient calculation involved in edge detection is sensitive to noise and the circle approximation does not yield high accuracy. On the other hand, as illustrated in Fig. 1, the transformation of each image from each location of the captured calibration board to the frontal image allows using the DIC concept to accurately locate the position of each control point by comparing the frontal image with the synthesized templates. Datta 4 achieved subpixel-location detection of the control points by carrying out a quadratic fitting in the neighborhood regions based on their correlation coefficients, but such a peak-finding approach is less accurate than the iterative schemes.8 Here, the proposed technique employs a high-accuracy cross-correlation algorithm, and the correlation coefficient function is:6, 8 Eq. 4[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{equation} C = \sum _{i=1}^N {\left[af(x_i,y_i)+b-g\big(x^{\prime }_i,y^{\prime }_i\big)\right]^2}, \end{equation} \end{document}Eq. 5[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{eqnarray} x^{\prime }_i &=& x_i + u + u_x (x_i-x_0) + u_y (y_i-y_0), \nonumber \\[-6pt]\\[-6pt] y^{\prime }_i &=& y_i + v + v_x (x_i-x_0) + v_y (y_i-y_0),\nonumber \end{eqnarray} \end{document}Fig. 1The conversion from raw images to frontal images enables the correlation with the ring templates. ![]() The rationale of the proposed method is that although the imprecise control points detected in the raw images lead to inaccurate camera calibration parameters, they offer good initial information about the relation between the camera and each scene which can be further processed to achieve very accurate localization of the control points. This information helps detect the camera intrinsic and extrinsic parameters as well as the world coordinates of the control points with high accuracies. The procedure of the proposed technique is summarized as follows:
To demonstrate the validity of the proposed technique, computer simulation along with real experiment have been conducted. The results suggest that no less than 40 control points and 15 positions are required to ensure a reliable calibration. Both simulation and experiment presented below use a flat panel with 10 × 7 ring patterns whose grid distance is 25.4 mm (as illustrated in Fig. 1), and 20 images at different positions are used in the calibration. In the simulation, the images are synthesized with camera parameters obtained from a real calibration where the radial, tangential, and prism lens distortion are considered. In addition, Gaussian noise with a standard deviation of 0.2% of the 25.4 mm distance is added to the position of each ring pattern in the x and y directions, and the images are also blurred and Gaussian noise with σ = 2.0 is added. Figure 2 shows the errors in locating the positions of the control points against their true values. Using Heikkila's technique7 to detect the control points directly in the raw images, the effect of perspective distortion where the ellipse center is not the actual circle center is obvious. In contrast, with the frontal image and DIC concepts, the distortion effect is removed, and over 20 times localization improvement in terms of the root-mean-square error (RMSE) in both x and y directions is achieved. The residual of the calibration, named reprojection error and defined as the RMSE between the projection of the control points to the image planes and their measured locations, is 0.00086 pixels. Table 1 shows the results of the intrinsic parameters. It can be seen that the camera parameters can be accurately retrived once the control points are detected with high precision. Fig. 2Localization errors of the control points obtained by Heikkila's method and the proposed method. ![]() Table 1Intrinsic calibration parameters.
For the real experiment, Fig. 3 shows the displacement error vectors between the detected and projected positions at each control point of the calibration target. With the conventional method and the proposed one, the reprojection errors are 0.3848 and 0.0098 pixels, respectively. Because of the insufficiency of the lens distortion model and the imperfection of the calibration target, the reprojection error is noticeably polarized with the conventional method in spite of its control refinement. With the proposed method, the error behaves more isotropic, indicating that the lens distortion and the imprecision of the calibration target have been successfully compensated. AcknowledgmentsThis work was supported by the Army Research Office (ARO) under Grant No. W911NF-10-1-0502 and the National Science Foundation (NSF) under Grant No. CMMI-0825806. ReferencesZ. Zhang,
“A flexible new technique for camera calibration,”
IEEE Trans. Pattern Anal. Mach. Intell., 22
(11), 1330
–1334
(2000). https://doi.org/10.1109/34.888718 Google Scholar
M. Pollefeys, R. Koch, and L. van Gool,
“Self-calibration and metric reconstruction in spite of varying and unknown intrinsic camera parameters,”
Int. J. Comput. Vision, 32
(1), 7
–25
(1999). https://doi.org/10.1023/A:1008109111715 Google Scholar
J. M. Lavest, M. Viala, and M. Dhome,
“Do we really need an accurate calibration pattern to achieve a reliable camera calibration?,”
158
(1998). Google Scholar
A. Datta, J. S. Kim, and T. Kanade,
“Accurate camera calibration using iterative refinement of control points,”
(2009). Google Scholar
D. Douxchamps and K. Chihara,
“High-accuracy and robust localization of large control markers for geometric camera calibration,”
IEEE Trans. Pattern Anal. Mach. Intell., 31
(2), 376
–383
(2009). https://doi.org/10.1109/TPAMI.2008.214 Google Scholar
M. A. Sutton, J. J. Orteu, and H. W. Schreier, Image Correlation for Shape, Motion, and Deformation Measurements, Springer, New York
(2009). Google Scholar
J. Heikkila,
“Moment and curvature preserving technique for accurate ellipse boundary detection,”
(1998). Google Scholar
B. Pan, K. Qian, H. Xie, and A. Asundi,
“Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review,”
Meas. Sci. Technol., 20 062001
(2009). https://doi.org/10.1088/0957-0233/20/6/062001 Google Scholar
|