Visually Guided Grasping

Motivation

Implement a reactive grasping system using state-of-the art technology. Bearing in mind that the more complete manipulation system known are among the human being capabilities, we are very interested to understand this biological behavior to incorporate new strategies for hand-eye coordination in the robot.

Abstract

Analytical approaches to the grasping problem can be discarded if we are looking for real systems working within unstructured environments, because the only possibility to make tractable these uncertainty domains will be by means of robot perception. So, our research is focus in the implementation of an integrated system for vision-guided grasping for service robots. By using very limited resources —a-state-of-the-art PC and an end-effector camera— an inexpensive robot arm can stably grasp unknown everyday objects in real time by means of visual information and visual feedback. Our system integrates computer vision, to capture the shape of the objects, on-line grasp determination based on that shape, and image-based control for grasp execution. Novel techniques are researched to solve the involved problems under the imposed resource constraints: namely, for information reduction and segmentation in image processing, strategies for grasp determination, as well as vision-guided control for grasp execution. Particularly, two different grasp determination strategies have been proposed until now: the former looks for optimal grips within a gravitational field, and actually tries to find the best solution, which, if it exists, will be unique. While the second strategy seeks to locate all the grips according to certain stability criteria, ignoring the gravitational field assumption. Both strategies are complementary in the sense that they endow a robot with different manipulation skills in complex, non-structured scenarios. And, finally, related to the grasping execution, a novel technique for sampling the visuomotor Jacobian is introduced. Nowadays, experimental validation results have been obtained showing how the robot arm can efficiently and stably grasp unknown everyday objects.

Results

Until now we have focused our research in static 2D scenarios.

Two different approaches are considered in order to get the grasp determination.

(A1) That follows an heuristic procedure to determine an optimal grip within a gravitational field. And,

(A2) That seeks to locate all the grips according to certain contact stability criteria, ignoring the gravitational field assumption.

---

(A1) Results

We can appreciate better how this procedure works looking at the following graphic results:
 
 

ai= 2.49/16.59, bi= 8.67/2.06, g12= 6.78	ai= 9.46/18.87, bi= 3.09/4.67, g12= 0.13	ai= 3.53/18.43, bi= 0.69/1.16, g12= 0.15
Toy salamander	Banana	Pepper

FIGURE 1. Some results of grasp determination for everyday objects. Note, in the lower row, that the grasp can be evaluated taking into account three parameters that are related to three thresholds: curvature (a), angular (b, between the normal and the grasping line), and distance (g, between the centroid and the grasping line).
 
 

FIGURE 2. The plot shows curvature for the pincers case. The smooth curvature conditions are satisfied (see the low values in the finger adaptation zone). In the top right figure the directions taken by the algorithm proposed trying to correct the angular conditions (bi) are displayed.
 
 

  FIGURE 3. The gripper in action: "unstable" grasp of pincers. The second thresholds are not satisfied and stability is not guaranteed. To understand in depth how the algorithm works making use of the namely "curvature-symmetry fusion" concept we remit to the reader to our recent article: [Sanz et al., 99]. Jointly with this latest concept we have introduce another parameter, namely "normalized global symmetry deficiency", , that permits evaluate the symmetry degree of a shape related to a given direction. See table 1 for instance.

	Imin			Imax
Shape		F1	s	F1	s
Nut		1.86	0.74	1.99	0.69
Screwdriver		1.58	0.82	4.25	0.57
Scissors		1.59	0.92	4.85	0.32
Screw		1.69	0.62	5.23	0.75
Pliers		2.68	0.54	4.67	0.83
Pincers		4.42	0.61	5.60	0.87
Caliper		5.12	0.30	7.19	0.64
Allen key		6.36	0.58	5.46	0.78

Table 1. (See table 2 for corresponding shapes). Normalized global symmetric deficiency F computed for different images in both directions Imin and Imax. It shows the mean, F1, and the standard deviation, s, for each one.
 

Nut	Screwdriver	Scissors	Screw
Pliers	Pincers	Calliper	Allen key

Table 2. The collection of shapes for the experiment described in Table 1.
Summarizing, F can be considered as another "descriptor" in pattern recognition techniques. [Sanz et al., 99]. Sanz PJ, Iñesta JM, del Pobil AP. "Planar Grasping Characterization Based on Curvature-Symmetry Fusion". Applied Intelligence, 10, 25-36, Ed Kluwer Academic Publishers, 1999.

---

(A2) Results
In this other approach we need introduce new concepts, such as "Grasping Region" or "Compatible Region".

FIGURE 4. In this figure it is shown how "grasping regions" are determined from the k-angular bending vector and their position on the object contour.
 
 

	A1	A2
Llave allen
Pincers
Figure by [1]

TABLE 3. Comparative analysis between the two proposed strategies. Summarizing, this second approach, A2, seems to be more flexible than the former, A1, and, ideally, include the grasp found by A1, among its set of grasps. The reader is remited to [CIRA'99] to gain a comprehension in depht about this strategy.

[1] Faverjon B, Ponce J. "On Computing Two-Finger Force-Closure Grasps of Curved 2D Objects". In Proc. IEEE Intl. Conf. on Robotics and Automation, 424-429, 1991.

Future Lines
Nowadays we became to extend our research towards 3D scenarios, primarily static and finally dynamic ones. To achieve these very complex goals we are collaborating with other experts in the hand-eye coordination domain, mainly in Germany (Technische Universität München).