Face detection and landmarks¶

Luna VL provides methods for faces detection on images and find landmarks on it.

Face detection¶

Detectors¶

There are 3 face detectors: FACE_DET_V1, FACE_DET_V2, FACE_DET_V3. FaceDetV1 detector is more precise and FaceDetV2 works two times faster. FaceDetV1 and FaceDetV2 performance depends on number of faces on image and image complexity. FaceDetV3 performance depends only on target image resolution. FACE_DET_V3 is the latest and most precise detector. In terms of performance it is similar to FaceDetV1 detector. FaceDetV3 may be slower then FaceDetV1 on images with one face and much more faster on images with many faces.

You should create a face detector using the method createFaceDetector of class VLFaceEngine for faces detection. You should set detector type when creating detector. Once initialize detector can be using as many times as you like.

Warning

We don’t recommend create often new detector because it is very slowly operation.

Face alignment¶

Face alignment is the process of special key points (called “landmarks”) detection on a face. FaceEngine does landmark detection at the same time as the face detection since some of the landmarks are by products of that detection.

Landmarks5¶

At the very minimum, just 5 landmarks are required: two for eyes, one for a nose tip and two for mouth corners. Using these coordinates, one may warp the source photo image for use with all other FaceEngine algorithms. All detector may provide 5 landmarks for each detection without additional computations.

landmarks5 contains the following landmarks:

Array index	Landmark location
0	Left eye center
1	Right eye center
2	Nose tip
3	Left mouth corner
4	Right mouth corner

Landmarks68¶

More advanced 68-points face alignment is also implemented. Use this when you need precise information about face and its parts. The 68 landmarks require additional computation time, so don’t use it if you don’t need precise information about a face. If you use 68 landmarks, 5 landmarks will be reassigned to more precise subset of 68 landmarks.

landmarks68 contains the landmarks according to I-Bug 68-point annotation scheme.

Examples¶

"""
Module realize simple examples following features:
    * one face detection
    * batch images face detection
    * detect landmarks68 and landmarks5
"""
import pprint

from lunavl.sdk.faceengine.engine import VLFaceEngine
from lunavl.sdk.faceengine.setting_provider import DetectorType
from lunavl.sdk.image_utils.image import VLImage


def detectOneFace():
    """
    Detect one face on an image.
    """
    faceEngine = VLFaceEngine()
    detector = faceEngine.createFaceDetector(DetectorType.FACE_DET_V1)

    imageWithOneFace = VLImage.load(
        url='https://cdn1.savepice.ru/uploads/2019/4/15/194734af15c4fcd06dec6db86bbeb7cd-full.jpg')
    pprint.pprint(detector.detectOne(imageWithOneFace, detect5Landmarks=False, detect68Landmarks=False).asDict())

    imageWithSeveralFaces = VLImage.load(
        url='https://cdn1.savepice.ru/uploads/2019/4/15/aa970957128d9892f297cdfa5b3fda88-full.jpg')
    pprint.pprint(detector.detectOne(imageWithSeveralFaces, detect5Landmarks=False, detect68Landmarks=False).asDict())

    imageWithoutFace = VLImage.load(
        url='https://cdn1.savepice.ru/uploads/2019/4/15/3e3593dc2fd0671c7051b18c99974192-full.jpg')
    pprint.pprint(detector.detectOne(imageWithoutFace, detect5Landmarks=False, detect68Landmarks=False) is None)


if __name__ == "__main__":
    detectOneFace()

Module contains function for detection faces on images.

class lunavl.sdk.faceengine.facedetector.BoundingBox(boundingBox)[source]¶

Detection bounding box, it is characterized of rect and score:

rect (Rect[float]): face bounding box

score (float): face score (0,1), detection score is the measure of classification confidence
and not the source image quality. It may be used topick the most “confident” face of many.

asDict()[source]¶

Convert to dict.

Returns: self.rect, “score”: self.score}
Return type: {“rect”
Return type: dict

rect¶

Get rect.

Returns: float rect
Return type: Rect[float]

score¶

Get score

Returns: number in range [0,1]
Return type: float

class lunavl.sdk.faceengine.facedetector.FaceDetection(coreDetection, image)[source]¶

boundingBox¶

face bounding box

Type: BoundingBox

landmarks5¶

optional landmarks5

Type: Optional[Landmarks5]

landmarks68¶

optional landmarks5

Type: Optional[Landmarks68]

_image¶

source of detection

Type: VLImage

asDict()[source]¶

Convert face detection to dict (json).

Returns: ‘rect’, ‘score’. optional keys: ‘landmarks5’, ‘landmarks68’
Return type: dict. required keys
Return type: Dict[str, Union[dict, list]]

image¶

Get source of detection.

Returns: source image
Return type: VLImage

class lunavl.sdk.faceengine.facedetector.FaceDetector(detectorPtr, detectorType)[source]¶

Face detector.

_detector¶

core detector

Type: IDetectorPtr

detect(images, limit=5, detect5Landmarks=True, detect68Landmarks=False)[source]¶

Batch detect faces on images.

Parameters

images – input images list. Format must be R8G8B8
limit – max number of detections per input image
detect5Landmarks – detect or not landmarks5
detect68Landmarks – detect or not landmarks68

Returns

return list of lists detection, order of detection lists is corresponding to order input images

Raises

LunaSDKException(LunaVLError.InvalidImageFormat) – if any image has bad format or detect is failed

Return type

List[List[FaceDetection]]

detectOne(image, detectArea=None, detect5Landmarks=True, detect68Landmarks=False)[source]¶

Detect just one best detection on the image.

Parameters

image – image. Format must be R8G8B8 (todo check)
detectArea – rectangle area which contains face to detect. If not set will be set image.rect
detect5Landmarks – detect or not landmarks5
detect68Landmarks – detect or not landmarks68

Returns

face detection if face is found otherwise None

Raises

LunaSDKException – if detectOne is failed or image format has wrong the format

Return type

Union[None, FaceDetection]

redect()[source]¶: todo: wtf Returns:

redetectOne()[source]¶: todo: wtf Returns:

setDetectionComparer()[source]¶: todo: wtf Returns:

class lunavl.sdk.faceengine.facedetector.ImageForDetection[source]¶

Structure for the transfer to detector an image and detect an area.

Attributes: image (VLImage): image for detection detectArea (Rect[float]):

detectArea¶: Alias for field number 1

image¶: Alias for field number 0

class lunavl.sdk.faceengine.facedetector.Landmarks5(coreLandmark5)[source]¶

class lunavl.sdk.faceengine.facedetector.Landmarks68(coreLandmark68)[source]¶