4.4.1.

Overview

The positional correlation between the rotational orientation and the origin of the camera’s coordinate system remains consistent throughout its motion. This stability underscores the necessity to define both the rotation axis and a point within the camera’s coordinate system for calibrating the rotation axis.

During the motion, the camera and laser line traverse a shared plane as the motor rotates. We designate the axis of rotation as $\mathbf{u}$, represented by the perpendicular vector of this plane. Given that a particular point within the camera-laser line system follows a 3D circular orbit as the motor moves, we establish a point on the rotation axis $\mathbf{u}$ by identifying the center of the circular trajectory traced by the camera coordinate system’s origin during this motion.²⁴ We designate $\mathit{C}$ as a reference point on the rotation axis. In tandem with a known angular change, it is possible to convert any 3D point situated within the camera’s coordinate system at a specific angular position $\varphi$ into a pre-defined camera frame, referred to as the absolute coordinate system. The initial angular position at which the system initiates motion is the reference for this absolute coordinate grid ($\varphi =0$).

This process involves defining $A_B^F$ as the coordinate of $A_B$ in the spatial framework $F$ and $R\varphi$ as the rotation matrix that rotates a point by $\varphi$ degrees from $M_0$ to $M_{\varphi }$. Assume that two frames, $M_0$ and $M_{\varphi }$, are acquired during the motor’s motion, as depicted in Fig. 6. $M_0$ serves as both the starting frame and the absolute coordinate system. Subsequently, $M_{\varphi }$ is the frame achieved following the motor’s rotation by $\varphi$ degrees. A process is proposed to transform the coordinates of any point within the $M_{\varphi }$ coordinate system to the absolute $M_0$ frame. Let us define a laser point $A$ within the $M_{\varphi }$ coordinate system, denoted as $A^{\varphi }$. We aim to determine its coordinates within the absolute coordinate system $M_0$.

Fig. 6

Correlation between a laser point within its associated coordinate system; the absolute frame is established.

Introduce $A^{\prime }$ as a virtual point of $A$ that satisfies Eq. (2). Let $C$ denote a point on the rotation axis. As vector math is applied to vector ${\mathbf{v}}_{\mathbf{3}}$, we ascertain that ${\mathbf{v}}_{\mathbf{3}}$ conforms to Eq. (3). The rotation matrix $R_{\varphi }$, which rotates frame $M_0$ by $\varphi$ degrees to $M_{\varphi }$, establishes the relationship between ${\mathbf{v}}_{\mathbf{5}}$ and $\mathbf{t}$ through Eq. (4). Consequently, Eqs. (2) and (4) enable us to deduce Eq. (5). A further application of vector math leads to Eq. (6). In culmination, the coordinates of laser point $A$ in the camera coordinate system $M_{\varphi }$ are re-expressed using Eq. (7) within the absolute coordinate system $M_0$. The expressions are given as

The 3D transformation relationship between point $A$ in a particular frame $M_{\varphi }$ and the absolute reference frame $M_0$ comprises three successive steps. First, there is a translation using the translation matrix ${\mathbf{t}}^{\mathbf{0}}$ from the given coordinate system $M_{\varphi }$ to a coordinate system with its origin positioned at point $C$ along the rotation axis. Subsequently, vector ${\mathbf{v}}_{\mathbf{3}}^{\mathbf{0}}$ is rotated by an appropriate angle via the rotation matrix $R\varphi$, resulting in vector ${\mathbf{v}}_{\mathbf{4}}^{\mathbf{0}}$. Finally, vector ${\mathbf{v}}_{\mathbf{4}}^{\mathbf{0}}$ is translated from point $C$ to the origin of the absolute frame, executed through an inverse translation operation²⁴

During the scanning process, we capture a series of consecutive frames. By converting all pixels from their current frame to a destination frame [as shown in Eq. (8)] and combining the results, we reconstruct the 3D scene.

In Eq. (8), the rotation matrix $R$ has nine hidden components and three additional hidden components in the translation vector. This means that we need to specify 12 variables. To reduce the complexity of this expression, we re-represent it using the definition of a rotation matrix around any axis $\mathbf{u}$^24,30 [as shown in Eq. (9)]. By substituting this rotation matrix into our original expression, we reduce the number of unknowns to 7:3 for the coordinates of the point on the rotation axis, 3 for the coordinates of the rotation axis vector, and 1 for the known rotation angle, which is given as

4.4.2.

Calibration of camera-motor

Our system maintains a consistent angle for each rotation step throughout the rotation axis calibration and scanning procedures. During the calibration phase, the camera captures an image of the calibration board at each angle $\theta$. Notably, the calibration board remains fixed during this calibration process. By placing the observation point at the origin of the calibration board’s coordinate system, it becomes evident that the camera’s coordinate system undergoes circular motion.

4.4.3.

Establish the camera’s origin coordinate within the calibration board

After calibrating the camera, we determine the transformation matrix for each camera pose. This matrix consists of a rotation matrix $R_C$ and a translation vector $T_C$. These values are calculated using the image of the calibration board and must satisfy the conditions outlined in Ref. 31, given as

The camera’s optical center is the origin of the camera coordinate system, so its coordinates are $(\mathrm{0,0},0{)}^T$. Given that we know the values of $R_C$ and $T_C$ from our previous calculations, we quickly determine the coordinates $X^W$ of the camera’s optical center in the calibration board coordinate system as

Each image of the calibration board provides us with the coordinates of the camera’s optical center at each motor angle. The collection of all of these points, denoted as $c=O_c,O_{C1},\dots ,O_{Cn}$, forms an arc in space, as illustrated in Fig. 7.

Fig. 7

Calibration board coordinate system and camera frame at different times.

4.4.4.

Calculate the direction vector in the calibration board’s coordinate frame

Once the coordinates of the camera’s optical center are determined for each motor angle, we utilize the least-squares method to estimate the plane $p$ that encompasses this set of points. This computation is carried out following Eq. (12).

The perpendicular bisector of any line segment formed by two points on an arc always passes through the center point of the 3D circle. Therefore, the point where two perpendicular bisectors intersect, created by any four points on the circular part, is at the center of the 3D circle. In other words, we can always determine the center of a circle from any set of points on it. Because any line in 3D space has infinitely many perpendicular bisectors on its orthogonal plane, we need to determine which one contains the circle’s center. This perpendicular bisector is defined as the line of intersection between the plane containing the arc and the vertical bisector plane of a line segment formed by two points on the arc

To determine the perpendicular plane for any pair of points $O_i$ and $O_{i+1}$ on the circle, we first define the normal vector as $n_i=\overrightarrow{O_i{O}_{i+1}}$. The point on the plane is then defined as the midpoint $K_i$ of the line segment connecting $O_i$ and $O_{i+1}$. We determine set $k$ of orthogonal planes for all points on the circle by randomly choosing two points from the collection of points on the arch. Each element in set $k$ and plane $p$ containing the arc form a corresponding median line. By finding the median line for all members in set $k$, we obtain a set of median lines m.

As shown in Fig. 8, any two perpendicular bisectors from the set of orthogonal $d$ always intersect at the center of the 3D circle. By repeating these steps for all perpendicular bisectors in $d$, we determine a set $c$ that represents the center of rotation for the circle. The coordinates of center point $\mathit{C}$ are obtained by taking the average coordinates of all points in $c$ after filtering out any noise.

Fig. 8

Procedure of estimating the center of the 3D circle.

4.4.5.

Determine the direction vector within the camera coordinate system

The rotation axis $\mathbf{u}$ and its associated point $C$, as defined in the earlier section, are established in the context of the calibration board coordinate system. However, because this coordinate frame varies with the position of the calibration board, it is not suitable for reconstruction during scanning. To overcome this limitation, we need to utilize the rotation information in the camera coordinate system as the relationship between them remains consistent.

By utilizing the transformation relationship delineated in Eq. (10), we convert plane $p$ from the calibration board’s spatial frame to that of the camera. Consequently, we derive its normal vector, the rotation vector. The process for obtaining the point on the rotation axis mirrors this methodology.

4.4.6.

Orientation optimization

The rotation axis and point on the axis obtained in the previous step are based only on calculation. We performed an optimization method to get the optimal calibration parameters by minimizing the geometrical error in the calibration system, given as

We define the geometrical error as the point-to-point distance error Eq. (13), which amalgamates the distance errors of all of the calibration board’s feature points from the first pose with those from the subsequent poses. Incorrectly selected calibration parameters directly impact the magnitude of this point-to-point distance error. Because the rotation axis is the normal vector of the camera’s trajectory plane, the rotation vector and translation axis must be mutually perpendicular. A penalty score is incorporated whenever these two parameters fail to meet the perpendicular condition to account for any deviations from the perpendicularity. Consequently, the optimization equation is formulated as Eq. (14), where $N$ denotes the number of feature points on the calibration board and BB is a constant influencing the penalty magnitude in cases in which the rotation axis and translation vector deviate from perpendicularity.²⁴

The optimization challenge outlined in Eq. (14) was addressed using a non-linear optimization approach. The MATLAB Optimization Toolbox was employed to solve these minimization problems.

The procedure for calibrating the axis of rotation is depicted in Fig. 9. The camera captures images of the calibration board at various motor angles. Leveraging the camera’s calibration parameters, we deduce each pose’s rotation matrix $R_C$ and translation matrix $T_C$. These matrices facilitate the computation of the camera frame origin’s coordinates in the calibration frame system. The rotation axis emerges as the plane’s normal vector by fitting these origin points to a plane. The point belonging to this axis, the center of the 3D circle, is calculated through geometric relationships. Subsequently, the rotation information becomes defined by the axis of rotation $\mathbf{u}$ and the point on axis $C$, drawing from the calibration board’s feature points obtained in the preceding process. An optimization process is undertaken to minimize discrepancies, aiming to determine the optimal values for $\mathbf{u}$ and $C$. The transformation matrices between the camera and absolute frame systems can be computed upon integration with the rotation angle.

Fig. 9

Pipeline of the camera-motor rotation axis calibration process.

5.1.1.

Optimization result and the effect of the accuracy of the rotation device to the calibration result

The results of the reprojection between the feature points of the first camera’s pose and those of the other camera’s pose before and after the optimization process are shown in Figs. 13(a) and 13(b). As can be seen, the reprojection was significantly improved.

We divide our calibration process into two main steps: the estimation of the rotation axis and rotation vector and the optimization process. As explained earlier, the results of the first step are based solely on the camera’s origin with respect to the calibration board coordinate system at different positions. The motor position is not used in this step. Therefore, it is evident that the accuracy of the rotation device does not affect its results. However, in the optimization process (and the scanning process after calibration), the rotation matrix is calculated based on the motor position or rotated angle, which is related to the motor’s rotation accuracy. Figure 14 shows the reprojection error of feature points after optimization in a case in which the accuracy of the rotation device is low. In this case, we assume that the motor rotates a larger angle than it would be with the error of 0 deg, 0.05 deg, 0.1 deg, …, 0.45 deg, and 0.5 deg.

Fig. 14

Relationship between rotation accuracy and reprojection error after optimization.

5.2.1.

Planarity of the plane

This experiment serves the purpose of evaluating the quality of the scanned surface. The criteria for establishing a plane’s attributes through planarity are outlined in Fig. 15(a).³² In our scenario, we leverage information from both the ideal and actual planes to gauge and appraise the level of planarity exhibited by the scanned plane. We used a calibration board as the scanning subject owing to its even surface. The scanning system was aligned with the first camera frame’s coordinate system, facilitating straightforward validation of the plane’s planarity. By scanning the calibration board, a 3D point cloud was generated. The equation of the plane was then determined through the application of the least squares method. To assess planarity, we computed the sum of squared distances between each point in the point cloud and the estimated plane equation. To enhance the accuracy, any outlier points in the scanning outcome were eliminated using point cloud software such as MeshLab.

Fig. 15

(a) Definition of the plane constraints considering planarity;³² (b) illustration of the ideal plane, real plane, and planarity of a plane; (c) result of our ideal plane and the estimated plane.

The distance from each estimated point to the ideal plane, illustrated in Fig. 15(b), was computed as

We consider the calibration boards in this experiment to be reference objects, as shown in Fig. 16(a). In this experiment, we scanned the calibration board in two parts: the first was from the beginning of the scanning process (from 0 deg) and the other was to the end (from some to 360 deg). This scanning result provided a more comprehensive check of the planarity of the entire scanning process. The scanned results are shown in Fig. 16(b). Because the color of the scanning object affects the scanning result, as shown in Fig. 16(c), only the white color area was used to perform the planarity check, as shown in Fig. 16(d).

Fig. 16

(a) Set up for the planarity experiment; (b) scanning result; (c) line scanning results from objects of different colors; (d) sample used in the planarity check.

The average and standard deviation of the distance between the ideal plane and the points on the actual plane are shown in Fig. 17(a). We compared our results with those of Lee et al.,²⁰ shown in Fig. 17(b), and found that our planarity check results were better.

Fig. 17

(a) Distance errors from the calibration board plane to every point in our system; (b) distance error from the checkboard plane to every point in Ref. 20.

5.2.2.

Accuracy check

To evaluate the precision of our scanning system’s measurements in terms of dimensions, we employed a calibration block as the reference object, depicted in Fig. 18(a). Figure 18(b) details the block’s specific dimensions, and Fig. 18(c) showcases the resultant scan output. Notably, the consistent height difference between the two planar sections of the block remains evident throughout the experiment.

Fig. 18

(a) Calibration block; (b) dimension of the calibration block; (c) scanned result of the calibration block; (d) illustration of the way to measure the distance between two planes.

To verify the accuracy of the calibration process, we computed the height difference of the calibration block using 3D coordinates derived from our 3D reconstruction methodology. This height difference signifies the separation between the two surfaces or planes associated with the calibration block. Addressing potential flatness irregularities, we employed the point-plane relationship to determine this separation, as visually depicted in Fig. 18(d).

In this context, let Po1 and Po2 denote two points corresponding to consecutive surfaces on the calibration block. Using the least squares technique, we derived two planes, p1 and p2, from these point sets, as shown in Fig. 18(d). Subsequently, let C1 and C2 symbolize the centroids of Po1 and Po2, respectively. We calculated projection points A and B, representing the projections of C1 and C2 onto planes p1 and p2, respectively. The distance between the two planes is defined as the average of the distances from point A to plane p2 and from point B to plane p1, given as

The average and error of the distances between the two planes are presented in Table 4. The standard value, mean error, and root mean square error (RMSE) of all distances are compared with the results of Zhu et al.³¹ in Table 5.

Table 4

Measurement accuracy at different distances.

Table 5

Comparison of system measurement accuracies.

5.2.3.

Scanning quality check

This experiment aims to evaluate the qualitative reconstruction of scanned results for objects of various shapes. As shown in Fig. 19(a), we examined multiple items, including a tennis ball and a cup. The results, presented in Fig. 19(b), demonstrate that our scanner produces high-quality scans.

Fig. 19

(a) Scanning workspace for the quality check; (b) scanning result of the quantity check.

In addition, we conducted another experiment to scan an object with complex features. We used a human statue, as shown in Fig. 20(a). The results in Fig. 20(b) demonstrate that our scanner produces high-quality scans even for objects with intricate details compared with the original scene.

Fig. 20

(a) Real human statue used for the quality check; (b) scanning result of the quantity check.