See the Unseen with Artificial intelligence: COVID-19 Story

Posted by Aleksa Radosavčević on May 13, 2020 12:24:35 PM
Find me on:

In every pandemic crisis throughout the history, analyzing data and numbers mattered. And the history isn’t short. In modern days, collected information material on crisis was analyzed upon the end of each pandemic, resulting in statistic reports, based on which key takeaways were drawn out. This knowledge accumulation is invaluable, and represents the foundation of modern epidemiology, setting up the rules and guidelines for handling pandemic crisis situations. On the other hand, there’s no doubt that AI provides an abundant arsenal of tools used to fight pandemic. 

For example, in pandemic crisis risk management, machine learning based simulations and reinforcement learning based agent models are used to simulate pandemic spreads in urban areas. Vaccine development, identification of population with risk factors or sensitive age structures, medical equipment and resources scheduling, critical population surveillance and control, diagnosis and treatment – AI plays an important role when dealing with all these challenges.

City AI – one of the largest ecosystems focused on applied artificial intelligence has organized the first Balkan AI videoconference in early April, where the impact of AI and its solutions on the world after COVID-19 has been discussed. We had a chance to hear some interesting opinions of industry leaders and experts from the Balkan region, like GM at Comtrade Digital ServicesViktor Kovačević, and CTO at airtDavor Runje. Many interesting topics were discussed, but judging by the audience questions, the main interest was the implementation of AI in medicine and remote sensing. 

Contactless Body Temperature Measurement via Computer Vision

Since increased body temperature was identified as one of the most relevant COVID-19 symptoms, quickly after pandemic outbreak many governments introduced body temperature measurements on public places, with the intense circulation of the population. Although there are different ways and instruments used to perform this task, perhaps one of the most widely implemented methods is based on computer vision and video processing.

Thermal body temperature cameras


Among various deep learning based implementations, Cheng X. et al. (2018) proposed a non-invasive measurement method, relaying on calculating Skin Sensitivity Index (SSI) values for determining subject’s body temperature. The underlying hypothesis behind usage of SSI index states that whenever the human body is exposed to thermal stimulation, consequently blood circulation will also change, which should be reflected in skin’s color and texture.

To compute SSI, authors have developed a methodology which first takes video on input, extracts each frame of that video, and segments region of interest (ROI) for each frame. In the next step, saturation channel S (differentiating between pale and dark version of a color) is extracted for each ROI image, and mean values of S are used to calculate SSI values for each subject.  In the next step, SSI values are used in training of deep neural networks. Authors developed two non-invasive measuring methods of skin temperature based on SSI and Deep Learning, NISDL1 and NISDL2:

NISDL1 architecture:

NISDL1 architecture

NISDL2 architecture:

NISDL1 architecture 2

Source for NISDL1 and NISDL2 figures:

Whereas both models are based on DenseNet-201 type of convolutional neural network (CNN) architecture, they differ in embedding maneuvers and kernel sizes. Nevertheless, authors reported superior results for both models, when compared to concurrent approaches, reporting an error between 0–0.25℅ ℃ for more than 50% of test sample images.

Chest X-ray Diagnosis and Inference

As one of the key features of COVID-19 virus is to attack respiratory system, after its outbreak the importance of having a reliable methodology for analyzing lung X-ray images has been brought to urgency. Not only it’s crucial to thoroughly determine whether patient’s X-ray image is indicating positive status, but it should be as prompt as possible, in order to prevent further virus dissemination. Tremendous amount of research so far has been conducted on application of different types of CNNs in medicine, from microscopic levels:


Microscopic level


to inferencing on blueprints of whole skeletons:


Human Activity Recognition Video Surveillance


Majority of the computer vision research on respiratory system topic is conducted on chest X-ray images, where CNN models are trained to assign the specific condition (disease) from the realm of respiratory conditions, or to simultaneously assign multiple conditions, due to the interconnectedness among some diseases. Some endeavors in this field are aimed to apply computer vision in a conjunction with techniques from natural language opus. Wang X. et al. (2017) combines deep CNN in multi-label set up of 8 diseases depicted below, together with natural language techniques such as DNorm - machine learning method for keyword recognition and normalization and MetaMap - biomedical text mining tool, applied on radiology reports on X-ray images written by doctors:

Chest X-ray diagnosis


Regarding specific COVID-19 CNN implementations, even in such a short time period of less than 3 months since the virus emerged, researchers already managed to react and propose the first, although not yet validated, implementation model architecture, tailored to recognize positive chest images. Wang L. et al. (2020) proposed COVID – Net, a novel approach characterized by a long-range connectivity between the layers in network’s architecture, specifically adjusted to COVID-19 positive case detection. The architecture was obtained by combining human driven domain knowledge with machine-driven design exploration (to learn more about automatically generating machine learning designs, read this blog). The network is making a heavy use of a lightweight residual projection-expansion-projection-extension (PEPX) design pattern:

PEPX pattern



Proposed model was developed on 13,800 CXR images across 13,725 patient cases, aiming to predict 3 states: non - COVID pneumonia (5,538 patients), normal or no pneumonia (8,066 patients) and COVID-19 (121 patients). Although trained on a highly unbalanced dataset, in terms of Positive Predictive Value metric, model achieved 96.4% when predicting positive COVID-19 patient cases, 94.7% when predicting non-COVID pneumonia cases, and 89.9% when predicting normal cases.


It’s definitely appealing to see that remote sensing technologies are getting their implementations in the fields other than surveillance.  Discussed work regarding non–invasive body temperature measurement is hopefully one of the many implementations to follow in medicine diagnostic.

On the other hand, presented work regarding implementation of CNNs in COVID-19 diagnostics should be taken into the account with special caution, as explained by authors, since the paper is still waiting to be validated and confirmed: it’s not meant to serve as any kind of standard COVID-19 check-up procedures, yet. However, after it gets validation of peer research audience, re-trained and re-evaluated on the more robust and balanced dataset sample (containing more than 121 COVID-19 positive cases, since more COVID-19 positive X-ray images will be available), it could become a very useful tool in respiratory medical check toolbox.



  1. Non-invasive measuring method of skin temperature based on skin sensitivity index and deep learning

Authors: Xiaogang ChengBin YangKaige TanErik IsakssonLiren LiAnders HedmanThomas OlofssonHaibo Li (2018)

  1. Automatic detection and counting of retina cell nuclei using deep learning

Authors:  S. M. Hadi HosseiniHao ChenMonica M. Jablonski (2020)

3.       Skeleton-based Human Activity Recognition for Video Surveillance

Authors: Ahmed Taha , Hala H. Zayed , M. E. Khalifa, El-Sayed M. El-Horbaty (2015)

 ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases

Authors: Xiaosong WangYifan PengLe LuZhiyong LuMohammadhadi BagheriRonald M. Summers (2017)

  1. COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images

Authors: Linda WangAlexander Wong (2020)

Topics: artificial intelligence, data science, machine learning, covid19, coronavirus