Objective

To create an object and face recognition website using AWS Rekognition API

Expected Outcome

To recognise the object or face of the users by the computer

Pre-requisite

Basic Knowledge in HTML, CSS, JavaScript, and Node.js
An AWS account with a credit card to unlock a free service trial

To build a website, different codes are necessary to construct a “frame”. Below are the required codes which support the implementation of AWS Rekognition Service for building the object and face recognition website.

Imgur

1. index.html

This is the “backbone” of the website. It lays out the content of your page.

html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link href="rekognition-style.css" rel="stylesheet">
    <link rel="shortcut icon" href="#">
    <title>Document</title>
</head>

<body>
    <header>
            <h1>AWS Rekognition</h1>
    </header>
    <div id="content">
        <aside>
                <form> 
                    <div id="upload-section">
                        <p>Upload Your Photo HERE!</p>
                        <input type="file" name="file" id="s3-file" onchange="saveFile(this);">
                    </div>
                    <div id="operate-section">
                        <p>Please select:</p> 
                        <select id="userOption">
                            <option value="" default>Select</option>
                            <option value="Labels" default>Labels</option>
                            <option value="Faces" default>Faces</option>
                        </select>              
                    </div>
                </form>

                <div id="recognize-section">
                    <button id="recognize" onclick="rekognize()" disabled>Start Recognize</button>
                </div>

                <div id="result">
                    <h2>Result</h2>
                    <div id="resultItem">
                    </div>
                </div>
        </aside>

        <main>
            <figure>
                <h2>Preview Uploaded Photo:</h2>
                <img alt="" id="img1" width="500px">
                <h2>Recognition Result:</h2>
                <div id="image-wrapper">
                    <img alt="" id="img2" width="500px">
                </div>
            </figure>
        </main>
    </div>
  <script src="script.js"></script>
</body>
</html>

2. style.css

This is for styling the website.

css

:root {
    --primary: #5628EE;
    --success: #41D071;
    --grey-light: #99A3BA;
    --grey: #6C7486;
    --grey-bg: #eee;
    --grey-dark: #3F4656;
    --light: #CDD9ED;
    --lighter: #E4ECFA;
    --wireframe-blue: #e9eeff;
    --btn-recognize: #ec7211;
    --btn-recognize-disabled: #eee;
    --shadow: rgba(18, 22, 33, .1);
}

body{
    width: 1000px;
    margin: 0 auto;
    background: var(--grey-bg);
}

body > div{
    /* border: 2px solid black; */ 
    display: flex;
}

header{
    background: #fff;
    /* border: 1px solid var(--lighter); */ 
    padding: 20px;
}

div#content{
    background: #fff;
    box-shadow: 0 4px 24px -2px var(--shadow);
    position: relative;
}

header > h1{
    text-align: center;
    margin: 0 auto;
}

aside{
    display: flex;
    flex-flow: column;
    align-items: stretch;
    /* border-right: 2px solid #000; */ 
    padding: 20px;
    flex: 3;
}

aside div{
    margin-bottom: 20px;
}

main{
    display: flex;
    padding: 20px;
    flex-flow: row wrap;
    flex: 7;
}

div#resultItem div.labels{
    display: flex;
    padding: 10px;
    flex-direction: row;
    border-bottom: 1px solid black;
    justify-content: space-between;
}

div#resultItem div.faces{
    display: flex;
    padding: 10px;
    flex-direction: row;
    border-bottom: 1px solid black;
    justify-content: space-between;
}

div#image-wrapper{
    position: relative;
    display:inline-block;
}

/* Bounding Box Related */ 

div.bounding-box{
    position: absolute;
    border-radius: 3%;
    border:2px solid rgba(255, 255, 255, 0.69);
    cursor: pointer;
}

.greybox{
    box-shadow: 0 0 1px #687078,0 0 2px #687078,0 0 2px #687078,0 0 2px #687078,inset 0 0 2px #687078,inset 0 0 2px #687078,inset 0 0 2px #687078,inset 0 0 2px #687078  !important;
}

.bluebox{
    box-shadow: 0 0 1px #3184c2,0 0 2px #3184c2,0 0 2px #3184c2,0 0 2px #3184c2,inset 0 0 2px #3184c2,inset 0 0 2px #3184c2,inset 0 0 2px #3184c2,inset 0 0 2px #3184c2  !important
}

.tooltiptext{
    visibility: hidden;
    width: 120px;
    background-color: black;
    color: #fff;
    text-align: center;
    border-radius: 6px;
    padding: 5px 0;
    /* Position the tooltip */ 
    position: absolute;
    z-index: 1;
}

div.bounding-box:hover .tooltiptext{
    bottom: 101%;
    left: 50%;  
    margin-left: -60px;
    visibility: visible;
}

div.bounding-box:active .tooltiptext{
    bottom: 101%;
    left: 50%;  
    margin-left: -60px;
    visibility: visible;
}

/* Form */ 

div#upload-section, div#operate-section{
    padding: 16px;
    background: var(--wireframe-blue);
}

div#upload-section > [type='file'] {
    width: 100%;
    height: 100%;
    cursor: pointer;
}

/* Button */ 

button#recognize{
    white-space: nowrap;
    width: 100%;
    border: none;
    padding: 20px;
    font-size: 16px;
    font-weight: 300;
    color: #fff;
    cursor: pointer;
    background: var(--btn-recognize);
}

button#recognize:disabled{
    white-space: nowrap;
    width: 100%;
    border: none;
    padding: 20px;
    font-size: 16px;
    font-weight: 300;
    color: #fff;
    background: var(--btn-recognize-disabled);
}

/* Select Menu */ 

#resultSelect{
    margin-bottom: 20px;
}

/* Mobile Responsive */ 

@media screen and (max-width: 800px) {
    body, body div{
        display: grid;
        max-width: 480px;
        min-height: 0px;
        min-width: 0px;
    }

    aside {
        grid-row: 1;
        border-right: none;
        border-bottom: 1px solid var(--light);
        min-width: 0;
    }

    main{
        grid-row: 2;
        min-width: 0;
    }

    img{
        display: flex;
        max-width: 100%;
    }
}

3. script.js

This is to enhance the functionality and interactivity of the website.

let upload, filename, imageWidth, imageHeight, faceResult;
let resultContainer = document.querySelector("#result");
let resultItem = document.querySelector("#resultItem");
let btn_recognize = document.querySelector("#recognize");
let boundingBox_container = document.querySelector("#image-wrapper");
let boundingBox = document.querySelector(".bounding-box");
//Initialization 

function init() {
  let content = document.querySelector("#result");
  content.style.display = "none";
}

window.addEventListener("load", () => {
  init();
});

//Main part 

function saveFile(event) {
  const xhr = new XMLHttpRequest();
  const formData = new FormData();
  const file = event.files[0];
  btn_recognize.disabled = true;
  formData.append("s3-file", file);
  // console.log(file); 

  filename = file.name;
  xhr.onreadystatechange = (state) => {
    if (xhr.readyState == 4) {
      console.log("Done!");
      btn_recognize.disabled = false;
    }
  };
  xhr.timeout = 5000;
  xhr.open("POST", "/upload-to-s3");
  xhr.send(formData);
}

//Rekognition 

function rekognize() {
  const option = document.querySelector("#userOption").value;
  const bucket = "eie2s02test";
  if (option == "Labels") {
    const s3params = JSON.stringify({
      Image: {
        S3Object: {
          Bucket: bucket,
          Name: filename,
        },
      },
    });
    detectLabels(s3params);
  } else if (option == "Faces") {
    const s3params = JSON.stringify({
      Attributes: ["ALL"],
      Image: {
        S3Object: {
          Bucket: bucket,
          Name: filename,
        },
      },
    });
    detectFaces(s3params);
  }
}

//Labels 

function detectLabels(s3params) {
  fetch("/start-rekognition-labels", {
    method: "POST",
    body: s3params,
    headers: {
      "Content-Type": "application/json",
    },

  })

    .then((res) => {
      //Clear Result Section 
      resultItem.innerHTML = "";
      boundingBox_container.innerHTML = `<img alt="" id="img2" width="500px">`;
      return res.json();

    })
    .then((result) => {

      // console.log(result); 
      const image = document.querySelectorAll("img");
      image[1].src = image[0].src;
      result.Labels.forEach((label) => {
        let name = label.Name;
        let confidence = label.Confidence.toFixed(1);
        let instance = label.Instances;
        if (confidence > 80) {
          resultContainer.style.display = "block";
          tempItem = `<div class="labels"> <span>${name}</span> <span>${confidence} %</span></div>`;
          resultItem.innerHTML += tempItem;
        }
        if (instance.length > 0) {
          //Add Bounding Box 
          instance.forEach((instance) => {
            let style = createBoundingBox(instance, imageHeight, imageWidth);
            //Add Tooltipbox 
            let tooltipbox = `<div class="tooltiptext">${name}</div>`;
            let container = `<div class="bounding-box greybox" style=${style}>${tooltipbox}</div>`;
            boundingBox_container.innerHTML += container;
          });
        }
      });
    })
    .catch((err) => console.log(err));
}
//detect faces 

function detectFaces(s3params) {
  fetch("/start-rekognition-faces", {
    method: "POST",
    body: s3params,
    headers: {
      "Content-Type": "application/json",
    },
  })
    .then((res) => {
      resultItem.innerHTML = "";
      boundingBox_container.innerHTML = `<img alt="" id="img2" width="500px">`;
      return res.json();
    })

    .then((result) => {
      console.log(result);
      const image = document.querySelectorAll("img");
      let menu = document.createElement("select");
      let counter = 1;
      faceResult = result.FaceDetails;
      image[1].src = image[0].src;
      menu.id = "resultSelect";
      if (faceResult.length > 0) {
        resultItem.appendChild(menu);
        //Append options to menu 
        result.FaceDetails.forEach((result) => {
          let option = document.createElement("option");
          option.value = counter;
          option.text = `Result ${counter}`;
          menu.appendChild(option);
          counter++;
        });
        //Display first Face details 
        let detail = faceResult[0];
        let detail_container = document.createElement("div");
        let age = detail.AgeRange;
        let gender = detail.Gender;
        let emotion = detail.Emotions;
        let eyeglasses = detail.Eyeglasses;
        let smile = detail.Smile;
        detail_container.innerHTML += `<div class="faces"> <span>Age Range</span> <span>${age.Low} - ${age.High}</span></div>`;
        detail_container.innerHTML += `<div class="faces"> <span>Gender</span> <span>${gender.Value}</span></div>`;
        detail_container.innerHTML += `<div class="faces"> <span>Emotion</span> <span>${emotion[0].Type}</span></div>`;
        detail_container.innerHTML += `<div class="faces"> <span>Smile</span> <span>${smile.Value}</span></div>`;
        detail_container.innerHTML += `<div class="faces"> <span>Eyeglasses</span> <span>${eyeglasses.Value}</span></div>`;
        resultItem.appendChild(detail_container);
        //Set Bounding Box for first Face 
        let style = createBoundingBox(faceResult[0], imageHeight, imageWidth);
        //Add Tooltipbox 
        let tooltipbox = `<div class="tooltiptext">1</div>`;
        let container = `<div class="bounding-box greybox" style=${style}>${tooltipbox}</div>`;
        boundingBox_container.innerHTML += container;
      } else {
        let message = document.createElement("p");
        message.innerText = "No Faces Found";
        resultItem.appendChild(message);
      }
      resultContainer.style.display = "block";
    })
    .catch((err) => {
      console.log(err);
    });
}

//Preview Photo 

upload = document.querySelector("#s3-file");
upload.addEventListener("change", () => {
  console.log("test");
  const image = document.querySelector("img");
  const file = document.querySelector("input[type=file]").files[0];
  const reader = new FileReader();
  if (file) {
    reader.readAsDataURL(file);
  }

  reader.onload = (e) => {
    image.src = e.target.result;
    image.onload = function () {
      imageWidth = this.width;
      imageHeight = this.height;
    };
  };
});

//show face details 

function displayFaceDetails(e) {

  if (e.target.nextElementSibling != null) {
    e.target.nextElementSibling.remove();
  }
  let detail = faceResult[e.target.value - 1];
  let detail_container = document.createElement("div");
  let age = detail.AgeRange;
  let gender = detail.Gender;
  let emotion = detail.Emotions;
  let eyeglasses = detail.Eyeglasses;
  let smile = detail.Smile;
  detail_container.innerHTML += `<div class="faces"> <span>Age Range</span> <span>${age.Low} - ${age.High}</span></div>`;
  detail_container.innerHTML += `<div class="faces"> <span>Gender</span> <span>${gender.Value}</span></div>`;
  detail_container.innerHTML += `<div class="faces"> <span>Emotion</span> <span>${emotion[0].Type}</span></div>`;
  detail_container.innerHTML += `<div class="faces"> <span>Smile</span> <span>${smile.Value}</span></div>`;
  detail_container.innerHTML += `<div class="faces"> <span>Eyeglasses</span> <span>${eyeglasses.Value}</span></div>`;
  resultItem.appendChild(detail_container);
}

function clearFaceBoundingBox() {
  const element = document.querySelector(".bounding-box");
  element.remove();
}

//bounding box parameters 

function createBoundingBox(instance, imageHeight, imageWidth) {
  let bot = instance.BoundingBox.Top * imageHeight;
  let right = instance.BoundingBox.Left * imageWidth;
  let height = instance.BoundingBox.Height * imageHeight;
  let width = instance.BoundingBox.Width * imageWidth;
  let style = `"top: ${bot}px; left: ${right}px; height:${height}px; width: ${width}px;"`;
  return style;
}

//Event 

document.addEventListener("mouseover", (e) => {
  if (e.target.className.split(" ")[0] == "bounding-box") {
    e.target.classList.add("bluebox");
  }
});

document.addEventListener("mouseout", (e) => {
  if (e.target.className.split(" ")[0] == "bounding-box") {
    e.target.classList.remove("bluebox");
  }

});

document.addEventListener("change", (e) => {
  if (e.target.id == "resultSelect") {
    displayFaceDetails(e);
    clearFaceBoundingBox();
    let style = createBoundingBox(
      faceResult[e.target.value - 1],
      imageHeight,
      imageWidth

    );
    let tooltipbox = `<div class="tooltiptext">${e.target.value}</div>`;
    let container = `<div class="bounding-box greybox" style=${style}>${tooltipbox}</div>`;
    boundingBox_container.innerHTML += container;

  }

});

Setting up the Environment

To implement the AWS Service, we are required to install 2 applications (node.js and aws-sdk-v3).

1. Install node.js

Go to https://nodejs.org/en/
Install the latest LTS Version.

2. Install aws-sdk-v3

Go to https://www.npmjs.com/package/aws-sdk.
In the middle of the page, you can see a command to install latest version of aws-sdk.
Copy the command to the terminal in VSCode / Command Prompt in Windows [Tip: Go to Search in Windows → type “Command Prompt” → paste the above command → click Enter]
VSCode Command Prompt:

3. Configuration

Go to package.json.
Change the content according to the image below. [Except the dependencies part]

Connecting to AWS Service

Refer to the API documentation of the AWS Rekognition, a total of 2 methods are offered to use with our own coding.

Directly connect to AWS Rekognition API.
Upload an image to the S3 bucket, then connect to AWS Rekognition API.

To get familiar with one more AWS Service, the S3 bucket, we have chosen the 2nd method for this task.

Obtain API key from AWS

Sign in to AWS as a root user. [Make sure your region is “Virginina” ]
Go to “Security credentials”.
Find “Access Key” and “Secret Access Key”.
Save them into a text file. [Tip: Download key File/ Copy and paste them to Notepad on your computer → save the file as TXT format]

Connect to the S3 Bucket

S3 bucket creation process
Search “S3” in the Search Engine
Go to “Create bucket”.
Input “Bucket name” then click “Create bucket”.
Go to the bucket page and upload a photo for testing.

S3.js

javascript

  import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
  import * as fs from "fs";
  import formidable from "formidable";

  // const formidable = import("formidable"); 

  const REGION = "us-east-1";
  const ACCESSKEY = "<First Key obtained in 3.1.4>";
  const SECRETKEY = "<Second Key obtained in 3.1.4>";
  const bucketName = "<Your S3 bucket name>";

  const S3 = new S3Client({
    region: REGION,
    credentials: {
      accessKeyId: ACCESSKEY,
      secretAccessKey: SECRETKEY,
    },
  });

  async function s3Upload(req, res) {
    const formData = await readFormData(req);
    try {
      await uploadFileToS3(formData.file, bucketName);
      res.send("Uploaded!");
    } catch (e) {
      console.log(e);
      res.send("Failed!");
    }
  }

  async function readFormData(req) {
    return new Promise((resolve) => {
      const dataObj = {};
      var form = new formidable.IncomingForm();
      form.parse(req);
      form.on("file", (name, file) => {
        dataObj.name = name;
        dataObj.file = file;
      });

      form.on("end", () => {
        resolve(dataObj);
      });
    });
  }

  async function uploadFileToS3(fileObj, bucketName) {
    const fileStream = fs.createReadStream(fileObj.filepath);
    const params = {
      Body: fileStream,
      Bucket: bucketName,
      Key: fileObj.originalFilename,
    };
    const uploadData = await S3.send(new PutObjectCommand(params));
    return uploadData;
  }
  export { s3Upload };

Combine all into AWS Rekognition

aws_rekognition.js

javascript

// Import required AWS SDK clients and commands for Node.js 
import {
  DetectLabelsCommand,
  DetectFacesCommand,
} from "@aws-sdk/client-rekognition";

import { RekognitionClient } from "@aws-sdk/client-rekognition";

// Set the AWS Region. 
const REGION = "us-east-1"; //e.g. "us-east-1" 
// Create SNS service object. 
const rekogClient = new RekognitionClient({
  region: REGION,
  credentials: {
    accessKeyId: "<First Key obtained in 3.1.4>",
    secretAccessKey: "<Second Key obtained in 3.1.4>",
  },
});

const detect_labels = async (params) => {
  try {
    const response = await rekogClient.send(new DetectLabelsCommand(params));
    return response; // For unit tests. 
  } catch (err) {
    console.log("Error", err);
  }

};

const detect_faces = async (params) => {
  try {
    const response = await rekogClient.send(new DetectFacesCommand(params));
    return response; // For unit tests. 
  } catch (err) {
    console.log("Error", err);
  }
};

export { detect_labels, detect_faces };

server.js

This is the backend of this project. All required parts are stored here.

javascript

import { s3Upload } from "./s3.js";
import { detect_labels, detect_faces } from "./aws_rekognition.js";
import express from "express";
import path from "path";
import { response } from "express";

const app = express();
const __dirname = path.resolve();

app.use(express.static(path.join(__dirname, "public")));
console.log(path.join(__dirname, "public"));

app.get("/", (req, res) => {
  res.sendFile(__dirname + "/index.html");
});

app.post("/upload-to-s3", s3Upload);
app.use(express.json());

app.post("/start-rekognition-labels", async (req, res) => {
  let result = detect_labels(req.body);
  result.then((response) => res.send(response));
});

app.post("/start-rekognition-faces", async (req, res) => {
  let result = detect_faces(req.body);
  result.then((response) => res.send(response));
});

const port = process.env.port || "8080";
app.listen(port, () => {
  console.log("App Listening on: " + port);
});

export { app };

File Location

With the implementation of express.js, it is required to define which file to be “public”. Therefore, there is a minor change in the file structure. i. Create a “public” folder. ii. Put “style.css” and “script.js” into the folder. Imgur

Testing

At the last stage of the project, we can now test and see if everything is functioning.

Type “node server.js” in the VSCode terminal.
Click Enter, and you will see the “App Listening on: 8080” message
Open any Browser. [Chrome would be a better option.]
Type “localhost:8080” at the URL bar.
Finally, you should be able to see a complete view of your project.

Introduction to AI

AI in Finance and Investment

AI in Medicine

Robots and Robotics

NLP and Machine Translation

AI in Games

Neural Networks and Deep Learning

Exercises

Videos

References

Exercises

Videos

References

Object & Face Recognition Website with AWS Rekognition API

Table of Content

Objective

Expected Outcome

Pre-requisite

Website Layout

1. index.html

2. style.css

3. script.js

Setting up the Environment

1. Install node.js

2. Install aws-sdk-v3

3. Configuration

Connecting to AWS Service

Obtain API key from AWS

Connect to the S3 Bucket

S3.js

Combine all into AWS Rekognition

aws_rekognition.js

server.js

File Location

Testing

Table of Content