What’s Your Vector Victor? Check Out SRI Sports’ Swing Diagnosis System

Recently SRI Sports (the parent company of Srixon and Cleveland Golf) was granted an interesting golf swing diagnosis system patent. The patent is USPN 7502491, which describes the invention as:

A computer (15) captures the colored moving image of the golfer (11) who swings by gripping a golf club (12) having colored marks (M1 through M3) attached to a shaft (12a) thereof. The computer (15) converts the colored moving image into a plurality of still images; executes binarization for each pixel of a plurality of the still images by using a specific threshold of color information and recognizes pixels, of the still images, which satisfy the threshold as positions of the colored marks (M1 through M3) and extracts a movement of the shaft (12a) by using a movement vector amount of one of the colored marks (M1 through M3 ) or by using a vector angle between two of the colored marks (M1 through M3). In this manner, the computer (15) automatically extracts check-point images.

Time to brush up on your pixel binarization to extract shaft motion using a movement vector.

The patent goes on to explain:

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a golf swing diagnosis system and more particularly to a system of automatically extracting check-point images effective for diagnosing a golfer’s swing form with high accuracy.

2. Description of the Related Art

There are proposed various kinds of apparatuses of photographing a golfer’s swing, automatically computing information such as the flight distance, orbit, and the like of a hit ball by a computer, and displaying the obtained information for a golfer. These apparatuses allow the golfer to examine the flight distance, orbit, and the like of the hit ball. However, these apparatuses are incapable of providing information useful for improving the golfer’s swing form.

In the swing form diagnosis apparatus disclosed in Japanese Patent Application Laid-Open No. 2003-117045, a golfer’s swing is photographed to extract images of only specific motion points important for diagnosing the swing form. More specifically, frames regarding the golfer’s swing motion are extracted from the golfer’s moving image photographed by the photographing means. Specific motion points during the swing motion are judged according to results of analysis of the moving partial images in the differential image between each frame and the reference image. A frame corresponding to each of the specific motion points is extracted to display the image.

An image at an impact time is important for diagnosing the golf swing. The position before the impact time, namely, the take-back and the neighborhood of the switch-over from the top position to the downswing and the position after the impact time are particularly important for diagnosing the golf swing. This is because the swing form cannot be corrected at the impact time, if the golfer has an improper swing in the neighborhood of the impact time. To examine the cause of the improper swing form at the impact time, it is necessary to extract a plurality of images of to-be-checked swing postures from images of the take-back time and in the neighborhood of the top position. Thereby it is possible to diagnose the swing form by taking many to-be-checked points of the swing posture in consideration.

However, in the swing form diagnosis apparatus disclosed in Japanese Patent Application Laid-Open No. 2003-117045, images at the take-back time are extracted by merely executing the differential processing between frames. Thus there is a high possibility that an image of a different position is erroneously extracted for a golfer. An image in which the shaft is horizontal at the take-back time is extracted by regarding the frame having a minimum in the X-direction deviation amount in the result of the differential processing as the frame to be extracted. However, golfers’ vertical and horizontal motions during a swing are quite different from each other. Thus in the case of a golfer having a take-back while the golfer is swaying (moves horizontally), a horizontal image of the shaft is extracted at a low degree of accuracy by merely considering the X-direction deviation amount during the take-back.

In the extraction of images at the downswing time and the follow-through time, frames a predetermined period of time before and after the impact-time image which is the reference-point image are extracted. Considering that golfers’ swing tempos are quite different from each other, it is impossible to extract an image of the same swing position (swing posture) for a plurality of golfers. Thus even if a golfer intends to improve her/his swing form at the time when the golf club shaft is horizontal by comparing her/his swing form with a swing form of a high-class player or that of a professional player, there is no guarantee that the extracted image of her/his swing form is the image at the time when the golf club shaft is horizontal. Therefore it is impossible to compare her/his swing form with that of the high-class player or that of the professional player in the same swing position (posture).

SUMMARY OF THE INVENTION

The present invention has been made in view of the above-described problems. Therefore it is an object of the present invention to provide a system of capable of precisely extracting still images of a swing position effective for diagnosing a golf swing from a moving image of a photographed golf swing.

To solve the above-described problems, in the first invention, there is provided a golf swing diagnosis system including a computer for capturing a colored moving image obtained by photographing a golfer who swings by gripping a golf club having colored marks attached to a shaft thereof. The computer has a means for converting the colored moving image into a plurality of still images; a means for executing binarization for each pixel of a plurality of the still images by using a specific threshold of color information and recognizing pixels, of the still images, which satisfy the threshold as positions of the colored marks so as to obtain coordinate data of each of the colored marks; an operation extraction means for recognizing a movement of the golf club shaft by using a movement vector amount of one of the colored marks computed based on the coordinate data of each of the colored marks or by using a vector angle between two of the colored marks; an image extraction means for selectively extracting still images necessary for diagnosing a golf swing from a plurality of the still images, based on data obtained by the operation extraction means; and an output means for outputting the extracted still image.

In the above-described construction, binarization is executed for the color information such as hue, saturation, and lightness by using the specific threshold corresponding to the color of each of the colored marks. Thereby it is possible to automatically recognize the pixel corresponding to each of the colored marks of the still image. Thus with reference to the movement vector amount of the colored mark computed based on the coordinate data of the colored marks or with reference to the vector angle between two of the colored marks attached to the golf club shaft, it is possible to recognize the swing motion and selectively extract a still image useful for diagnosing the golf swing
from a plurality of the still images. The extracted still image useful for diagnosing the golf swing is outputted to a computer through a network or the like so that it is displayed on its display screen, printed, stored by an external recording medium. Thereby it is possible for a professional player (teacher) and a golfer to diagnose the golf swing by observing output results.

The area range regarded as the colored mark is set in an image in advance so that when the number of aggregates of pixels satisfying the binarized threshold is more than that of the colored marks, the aggregates of pixels falling in the set area range should be decided as the colored mark. Further by executing the differential processing between the colored mark and the background image, the processing of deciding the colored mark may be executed based on the area range after a region in which the colored mark is not present is cut.

It is preferable that the golf swing diagnosis system has a photographing means for photographing a golfer who swings by gripping the golf club shaft to which the colored marks are attached, thus providing a colored moving image.

It is preferable that the extracted still images necessary for diagnosing the golf swing are check-point images including an impact image and one or more swing postures other than the impact image.

The operation extraction means automatically traces a position of each of the colored marks of each still image by storing automatically recognized color information of red, green, and blue of each of the colored marks of one still image as reference color information; sets an allowable range of a color regarded as the same color as the reference color; sets on a subsequent frame a search range which is a region including an estimated position of each of the colored marks in a still image of the subsequent frame adjacent to the one still image in time series; and regards pixels falling in the color range as the positions of the colored marks in the search range.

The automatic tracing can be executed by the binarization or in combination of the color range and the binarization in addition to the method of using the color range.

In the above-described construction, to detect the position of each of the colored marks, whether a region falls within the color range is judged not in the entire screen but only in the search range. Therefore when the still image includes a color proximate to that of the colored mark, it is possible to eliminate the color, prevent an erroneous recognition, and shorten a computing period of time.

Differential processing is executed between the pixel within the search range of the still image and the background image to eliminate the background image. Thereby even if the background image includes the color proximate to that of the colored mark, it is possible to eliminate the color and prevent the erroneous recognition to a higher extent. The erroneous recognition can be prevented by carrying out a method in which the size of the area of the colored mark and the shape of the colored mark are considered.

When the colored marks cannot be traced, binarization is executed again on each pixel in the search range by using the specific threshold of the color information to obtain coordinate data by regarding pixels satisfying the threshold as the positions of the colored marks.

In the above-described construction, even if there is a still image in which tracing of the colored marks have failed, it is possible to obtain the coordinate data of the colored marks by executing binarization again.

The operation extraction means extracts a swing posture by using a movement vector amount between still images, of one of the colored marks provided on the shaft, adjacent to each other in time series. The image extraction means extracts a still image at an impact time and one or more images of a swing posture selected from among a take-back shaft 9 o’clock image, a top image, a downswing shaft 9 o’clock image, a follow-through shaft 3 o’clock image, and a finish image as check-point images.

In the above-described construction, a frame in which a Y-direction component of the movement vector amount of one colored mark is minimum is regarded as the impact image, and a frame in which an X-direction component of the movement vector amount of the colored mark is minimum is regarded as the take-back shaft 9 o’clock image. Thereby it is possible to selectively and automatically extract each check-point image useful for diagnosing the swing. It is to be noted that the longitudinal direction in the image is set as the Y-direction and that the lateral direction therein is set as the X-direction.

The operation extraction means extracts a swing posture by using a vector angle between two or more of the colored marks provided on the shaft at certain intervals or/and a movement vector amount of one of the colored marks near a grip. The image extraction means extracts a still image at an impact time and one or more images of a swing posture selected from among a take-back shaft 9 o’clock image, a top image, a downswing shaft 9 o’clock image, a follow-through shaft 3 o’clock image, and a finish image as check-point images.

In the above-described construction, a frame in which the vector angle is 90 degrees (horizontal) is regarded as the take-back shaft 9 o’clock image, and a frame in which the vector angle is 0 degree (vertical) is regarded as the impact image. Thereby it is possible to selectively and automatically extract each check-point image useful for diagnosing the swing.

The operation extraction means executes differential processing of the still image by using a background image in which a golfer is not photographed to obtain a golfer’s silhouette; extracts a contour of the silhouette; regards a pixel which makes a curvature of the contour extreme as an unskillful arm side shoulder; computes a position of the grip from a positional relationship between two of the colored marks; stores at least one part of the still image in a range from the shoulder at the unskillful arm side to the grip as a template; and extracts a movement of a golfer’s unskillful arm by executing template matching processing for a still image during a take-back swing. The image extraction means regards a frame of the still image in which the template has become horizontal as an image in which the unskillful arm is horizontal in the take-back swing, based on data obtained by the operation extraction means, thus extracting the still image as a check-point image. It is possible to execute the matching processing at the template angle in which starts the template matching processing without extracting silhouette from the angle as prescribed and memorized.

In the above-described construction, it is possible to recognize the angle of the matched template by template matching processing as the angle of the unskillful arm and automatically extract an image in which the golfer’s unskillful arm at the take-back time is horizontal. If there are two or more golfer’s silhouettes as a result of execution of the background subtraction between the still image and the background image, i.e., if another silhouette is erroneously extracted in addition to an actual silhouette, it is preferable to set an area range of the image which is considered the golfer’s silhouette in advance to determine a silhouette disposed in the area range as the golfer’s silhouette.

The image extraction means stores at least one part of the images in the range from the shoulder to the grip in the still image in which the unskillful arm is horizontal in the take-back swing as a template; and executes template matching processing for the still image in a downswing and regards a frame of an image that matches the template to a highest extent as the still image in which the unskillful arm is horizontal in the downswing, thus extracting the image that matches the template to the highest extent as a check-point image.

That is, by utilizing the fact that the un
skillful arm which is horizontal at the take-back time and the unskillful arm which is horizontal at the downswing time have almost the same state, the image extraction means stores at least one part of images in the range from the shoulder to the grip in the still image in which the unskillful arm is horizontal in the take-back swing as the template and executes the template matching processing for the still image in the downswing by using the template. Thereby it is possible to automatically extract the frame of the image in which the unskillful arm is horizontal in the downswing.

The image extraction means executes differential processing for the still image by using a background image in which a golfer is not photographed to obtain the golfer’s silhouette and obtain one end of the silhouette as a side of a golfer’s leg at the skillful arm side from a left-to-right width of the silhouette; and regards an image at a time when a perpendicular passing through the side of the leg at the skillful arm side intersects with the colored mark attached to the shaft as a take-back shaft 8 o’clock image, thus extracting the take-back shaft 8 o’clock image as a check-point image. The perpendicular may not be automatically extracted but a line stored in advance may be set as the perpendicular. A tester may be requested to take a stance by placing her/his skillful leg at a fixed position. Thereby the perpendicular to the skillful leg can be fixed without executing image processing.

In the above-described construction, by merely using the golfer’s silhouette and the coordinate data of each colored mark after the differential processing is executed, the take-back shaft 8 o’clock image can be automatically extracted.

The image extraction means regards a photographed first swing image as an address image or regards a still image in which a differential is minimum when differential processing is executed between frames from a start time of an extraction of the swing images as the address image, thus extracting the address image as a check-point image.

In the above-described construction, the image at the address which is the most important position in the golf swing can be automatically extracted. When a sound generated at an impact time and a signal outputted from an impact sensor are obtained as a trigger signal and when a moving image a predetermined period of time before and after the impact time is obtained, the first image is not necessarily the address image. Thus in this case, differential processing is executed between frames. A frame having a minimum differential is considered the state in which the golfer is stationary and regarded as the address image.

Cool stuff! It is just a matter of time before all golfers have access to low cost self-service swing diagnosis systems.

Dave Dawsey  – Tracking Golf Swing Monitoring Patents

PS – click HERE to check out interesting putter posts