Unlocking Real-World Potential: The Dynamics of Pipe-lined Models

Pipeline model refers to the organized orchestration of multiple models in a sequential manner to achieve a specific objective. The concept of pipelined models involves chaining together multiple independent models, creating a sequential workflow where the output of one model serves as the input for the next.
Here are some of the examples:
Face Detection + Gender/Age Classification: - The initial stage involves a face detection model, which identifies and localizes faces within an image or video stream.
Following face detection, the pipeline employs a Gender/Age classification model that predicts the gender and age of each detected face.
In a retail setting, this pipeline can be extended to counting the number of people(footfall), and the other demographics.
License Plate Detection + Optical Character Recognition (OCR) - In this scenario, a model specialized in license plate detection is employed to locate license plates within images or video frames. Following successful license plate detection, an OCR model is applied to recognize and extract the alphanumeric characters from the license plate.
Text Extraction + Text Summarization - This pipeline involves extracting relevant information from a large text using a text extraction model. The extracted content is then fed into a text summarization model to generate concise summaries.

Each stage of the pipeline is dedicated to a specific subtask, and models within each stage can be fine-tuned and optimized independently for their designated task. Pipelines are easily scalable as they can be extended with additional stages to address new tasks.The modular nature of the pipeline allows for easy replacement or updating of individual models without affecting the entire system.

In the realm of retail analytics, pipelined models can offer a comprehensive approach to understanding customer behavior and optimizing business operations.
Let's further explore the use case and advantages of pipelined models in the retail analytics domain.

Revolutionizing Retail Analytics

In the ever-evolving landscape of Retail, staying ahead of consumer trends and preferences is paramount for success. One of the groundbreaking technologies transforming the retail space is the integration of Face detection and Gender classification models. This powerful duo utilizes computer vision to not only identify individuals but also categorize them by age, gender and other demographic information, offering retailers unprecedented insights into their customer base.
As the retail landscape continues to evolve, businesses that embrace these technological advancements are poised to stay ahead of the curve, delivering more personalized and targeted services to their diverse customer base. The synergy between face detection and gender classification is not just a technological marvel; it's a strategic imperative for retailers aiming to thrive in the digital age.

• Face Detection in Retail:
  The first component of this dynamic duo is face detection. Using advanced computer vision algorithms, retailers can automatically detect and locate human faces within images or live video streams. This capability is particularly useful in crowded retail spaces, where traditional methods of data collection may fall short. Face detection technology enables retailers to track customer movement, measure foot traffic, and analyze engagement with specific products or store sections.
• Gender and Age Classification Models Enhancing Customer Understanding:
  Building upon face detection, the integration of Age and Gender classification models add another layer of intelligence to the retail analytics toolkit. These models leverage facial features to categorize customers by gender and estimate their age groups. Understanding this distribution of store visitors allows retailers to tailor their inventory, marketing strategies, and store layouts to better suit their diverse customer base.
• Enhancing Customer Experience:
  The integration of Face detection and Age/Gender classification models contributes to a more personalized and efficient customer experience. Imagine a scenario where a customer walks into a retail store, and the system detects their face, instantly categorizing their age/gender. This information can trigger personalized marketing messages or promotions, creating a more engaging and tailored shopping journey.
• Data-Driven Decision Making:
  Retailers can leverage the data collected by these models to make informed decisions. Insights into customer demographics, peak shopping hours, and popular products can guide inventory management, staffing levels, and marketing campaigns. This data-driven approach empowers retailers to optimize their operations and enhance customer satisfaction.
• Addressing Privacy Concerns:
  While the benefits of face detection and gender classification in retail are evident, it's crucial to address privacy concerns. Implementing robust security measures, anonymizing data, and obtaining customer consent are essential steps in ensuring that this technology is deployed ethically and in compliance with privacy regulations.

One of the significant components of retail analytics is Digital signage with Facial Recognition. Retailers leverage digital signage not only for advertising products but also for enhancing the overall shopping experience and collecting valuable data.

Here's how digital signage contributes to retail analytics:

• Customer Engagement: Interactive displays can capture attention, showcase products, and provide relevant information.
• Demographic Data Collection: With the integration of technologies like facial recognition, digital signage can gather real-time demographic data (For example, Age and Gender). Retailers use this data for targeted marketing and to understand their customer base better.
• Content Personalization: Retailers can use analytics to tailor content displayed on digital signage based on customer demographics. For example, an advertisement or promotion can be adjusted dynamically to appeal to specific age groups or genders.
• Inventory Management: Digital signage integrated with analytics can provide real-time information on product availability and inventory levels. This helps in managing stock efficiently and ensures that customers are informed about product availability.
• Feedback and Surveys: Interactive digital signage can be used to collect customer feedback through surveys or interactive elements. This data contributes to understanding customer preferences, satisfaction levels, and areas for improvement.

By combining digital signage with analytics, retailers gain insights into customer behavior, preferences, and the overall effectiveness of their in-store communication strategies. This data-driven approach enables retailers to make informed decisions, enhance customer experiences, and drive business success.

DeGirum offers a Python Software Development Kit, named PySDK, coupled with the cloud-based infrastructure provides you an easy way in integrating multiple models, each specialized in different aspects, the pipeline can leverage the strengths of each model, resulting in enhanced overall performance.
This approach is particularly beneficial when dealing with complex tasks or diverse data types. Additionally, the modularity of a two-model pipeline facilitates easier maintenance, updates, and scalability, providing a flexible solution to evolving challenges in various domains.

Compound models refer to the integration of two or more independent models to create a unified and more robust system. These models work collaboratively, with the output of one model influencing the input or processing of another.
We also provide a compound model class that integrates two distinct models:

• The first model, designed for Object Detection, identifies regions of interest within the original image (Face detection for the Face model)
• Subsequently, the identified regions are cropped[^1] (Inflated bounding box coordinates to crop extent and adjust to image size) and fed into the second classification model (Gender model / Race model / Age model), which adjusts bounding box labels according to classification results.
  [^1]:Face crops enhance computational efficiency and maintain consistency in input size by focusing solely on the critical facial details required for accurate gender prediction.
• The compound model delivers the final result based on the outcomes of the second model.

Steps to use the Pipeline Model (Face Detection Model + Gender Classification Model) using Degirum PySDK : <br>

1. Install DeGirum PySDK and DeGirum Tools Python packages

!pip show degirum || pip install degirum
!pip show degirum-tools || pip install degirum-tools

2. Define the name of the model zoo and model names (Face detection model + Gender classification model) from the DeGirum cloud public model zoo

hw_location = "@cloud"
face_model_zoo_url = "<https://cs.degirum.com/degirum/ultralytics_v6>"
face_model_name = "yolov8n_relu6_face--640x640_quant_n2x_orca1_1"
gender_model_zoo_url = "<https://cs.degirum.com/degirum/openvino>"
gender_model_name = "mobilenet_v2_gender--160x160_float_openvino_cpu_1"

The Gender model is trained on UTKFaces, FairFace and CelebA Open-source datasets.[^2]
[^2]:We cleaned the dataset by filtering out the face images with Age category (1-12) from both the UTKFaces and FairFace datasets, as many facial features that contribute to gender identification may not be fully developed or distinguishable at this stage.

3. Define a video source or an image source

image_source = "<https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/ThreePersons.jpg>"

4. Connect to the model zoo and load the model

gender_model = gender_zoo.load_model(gender_model_name)
face_zoo = dg.connect(hw_location, face_model_zoo_url, degirum_tools.get_token())
gender_zoo = dg.connect(hw_location, gender_model_zoo_url, degirum_tools.get_token())
face_model = face_zoo.load_model(face_model_name)
gender_model = gender_zoo.load_model(gender_model_name)

5. Define a compound cropping model with a crop extent

Earlier we used to run each model seperately (one after the other).
Firstly, Face detection model which outputs the face crops along with bounding box coordinates, followed by the Gender model that runs on the face crops.

face_zoo = dg.connect(hw_location, face_model_zoo_url, degirum_tools.get_token())
gender_zoo = dg.connect(hw_location, gender_model_zoo_url, degirum_tools.get_token())
with face_zoo.load_model(face_model_name as face_model:
with gender_zoo.load_model(gender_model_name) as gender_model:
# adjust some model properties
face_model.overlay_font_scale = 1.5
face_model.overlay_alpha = 1
face_model.image_backend = gender_model.image_backend = 'pil'

# detect faces
detected_faces = face_model(image_source)

# for each detected face
for face in detected_faces.results:
# recognize the gender inside the box
predictions = gender_model(detected_faces.image.crop(face['bbox']))
predicted_gender = max(prediction.results, key=lambda x: x['score'])
# change object class label to recognized number for nice display
face['lable'] = predicted_gender['label']
# display combined result
display(detected_faces.image_overlay)

We run a for loop over each faces and each face crop is fed as the input to the Gender classification model.

Now, we have build a Compound model that combines the two independent models into one pipeline. This model class takes in two models and the crop extent (default=50.0) as the parameters.
Note : Make sure the first model is of object detection type.

crop_model = degirum_tools.CroppingAndClassifyingCompoundModel (
face_model, gender_model, 50.0
)

6. Inference and display the output image along with its Gender

crop_output = crop_model(image_source)
img = crop_output.image_overlay
IPython.display.display(PIL.Image.fromarray(img[..., ::-1]), clear=True)

Sample output:

Explore the entire code here:

• With Image source
  Face-Gender prediction model in this Google Colab notebook.
• With Video source
  Face-Gender prediction model in this Google Colab notebook.