構建可以檢測潮熱并觸發動作來緩解人的設備

描述

概述

潮熱是上半身突然感到溫暖，通常在面部、頸部和胸部最為強烈。盡管其他醫療條件也可能導致潮熱，但潮熱最常見的原因是更年期。在這個項目中，我構建了一個可以用來檢測潮熱并觸發一些動作來緩解人的設備，在這種情況下，使用紅外發射器打開空調冷卻系統。作為輸入，它采用多維紅外熱傳感器數據。它的輸出將是一個簡單的分類，如果有人被識別并且最近發生了突然的溫度變化，它會通知我們。

硬件考慮

我想做一個可以讀取體溫數據的可穿戴設備。在做了一些研究和實施之后，我發現一直佩戴傳感器，尤其是在睡覺時，在很多情況下都是不舒服的，而且是不可行的。潮熱也會導致出汗，佩戴電池供電的設備需要特殊的外殼才能準確測量數據。此外，對于冷卻，我研究了熱電 Peltier 冷卻器模塊，發現它們中的大多數都需要更大的 12V 電池才能正常運行以實現最佳冷卻，如果不仔細控制它也是不安全的并且可能導致灼傷。對于非侵入式解決方案，我發現低分辨率熱像儀可以從遠處可靠地讀取溫度，并且它們可以在黑暗中工作。雖然讀取精度可能會受到距離的影響，但通過計算可以準確記錄熱圖像數據的連續或突然變化。我選擇 Atom Matrix 作為微控制器設備，因為它體積小、價格便宜、可以運行 TensorFlow 模型并且有一個可以用作遙控器的 IR 發射器。我選擇 Wio Terminal 來捕獲熱像儀數據進行訓練，因為它有一個帶有許多按鈕的 LCD 屏幕，用于捕獲不同類別的數據。

訓練數據收集

機器學習項目的第一步也是最重要的一步是以這樣一種方式收集訓練數據，即它應該涵蓋給定識別任務的大多數代表性案例。Wio 終端上的 3 個按鈕用于標記 3 個類別。

捕獲的數據將保存到連接到 Wio 終端的微型 SD 卡上的文件中。每個熱圖像數據都被捕獲為一個單獨的文件。該文件不包含標題行，只有逗號分隔的 768 (24x32) 溫度讀數。示例讀數文件內容如下所示。

26.47,25.97,25.85,25.72,26.90,26.12,26.60,26.86,27.00,26.68,26.90,26.74,27.78,27.21,27.75,29.12,31.29,31.50,32.24,31.95,31.72,30.80,31.29,30.69,31.18,30.86,31.46,31.37,29.21,28.23,28.18,28.03,25.83,26.33,25.55,26.56,26.59,26.90,26.52,27.38,26.94,27.39,26.85,27.21,27.32,27.66,28.81,30.45,30.97,31.74,31.55,32.14,31.37,31.03,30.63,30.69,31.03,31.52,30.85,31.14,28.80,28.57,27.81,28.39,26.43,26.24,26.67,26.71,27.13,26.99,27.63,28.07,28.59,28.39,27.80,28.19,28.11,28.25,30.91,32.15,31.78,31.46,31.82,31.33,31.10,30.43,30.37,30.06,29.77,29.84,30.58,30.45,29.28,28.42,28.34,27.76,26.31,26.62,26.38,27.24,27.27,27.91,28.94,29.11,30.11,29.73,30.25,29.53,29.59,29.22,32.01,32.70,33.17,32.00,31.15,31.52,30.59,30.46,29.87,30.07,29.43,30.09,30.01,30.68,29.10,28.91,27.99,28.34,26.59,26.60,26.99,27.49,28.68,29.64,31.88,33.14,33.41,33.02,32.48,32.83,32.60,32.60,33.58,34.16,34.79,34.58,32.43,32.15,31.07,30.77,29.84,29.89,30.01,29.48,29.99,29.63,29.33,28.47,28.23,27.90,26.26,26.26,27.13,28.21,30.50,31.41,33.23,33.70,33.44,33.40,33.38,33.18,33.31,33.12,33.65,34.33,34.81,34.93,33.96,32.50,31.29,30.82,29.37,29.93,29.13,29.93,29.29,30.07,28.76,29.00,28.38,28.69,26.83,26.55,27.36,28.68,32.50,33.12,33.78,33.40,34.17,34.14,33.80,33.56,33.84,33.35,33.85,33.54,34.63,34.45,34.68,34.49,31.76,30.94,29.87,29.38,29.67,29.20,29.45,29.68,29.02,28.42,28.63,28.49,26.29,27.30,27.51,28.71,31.96,33.70,33.86,33.76,33.87,34.32,33.60,33.94,33.41,33.39,33.74,34.04,34.32,34.95,34.24,34.63,31.81,31.15,29.59,29.78,29.19,29.62,28.77,29.72,28.88,29.12,28.51,28.79,26.45,27.15,28.06,28.72,32.45,33.05,33.89,33.84,33.60,33.39,34.02,33.69,33.64,33.29,34.11,33.81,34.68,34.55,34.75,34.15,32.55,31.42,30.25,29.76,29.56,29.42,29.64,28.59,28.87,28.66,29.10,28.79,26.76,27.06,27.39,28.97,31.63,33.46,33.78,34.22,33.97,34.11,33.58,34.20,34.01,33.97,33.69,34.16,34.54,34.90,34.33,34.63,33.09,31.81,29.81,30.15,28.89,29.81,28.96,29.39,28.70,29.35,28.54,29.07,27.18,26.59,27.31,27.98,31.52,32.56,33.37,33.52,33.72,33.72,33.52,33.22,33.75,33.27,33.93,34.13,34.69,34.65,34.26,34.12,32.70,31.12,30.58,30.45,29.91,29.42,29.72,28.82,29.66,29.10,28.91,29.04,26.26,26.72,26.66,27.70,29.47,31.74,32.43,33.58,33.48,33.45,32.63,32.79,31.94,33.32,33.36,34.14,34.42,34.55,33.99,33.87,31.15,30.99,30.15,30.84,29.81,30.02,29.24,29.61,29.15,29.18,28.82,29.32,27.21,26.60,27.01,26.83,27.83,27.64,29.27,29.08,30.97,29.96,29.96,28.30,29.66,30.77,34.03,33.60,34.23,33.26,32.36,31.42,30.61,30.63,30.44,30.50,30.45,30.46,29.70,29.37,29.70,29.34,29.17,28.83,26.28,26.53,26.36,27.31,26.68,27.55,27.46,28.27,27.83,28.53,27.72,28.01,28.14,30.84,32.27,33.38,32.09,32.70,31.33,31.32,30.03,30.53,29.86,30.86,30.35,30.88,29.78,29.66,29.58,29.62,29.06,29.77,26.63,26.16,26.65,26.85,27.10,26.79,27.12,26.67,27.47,27.14,27.15,27.15,27.93,28.86,31.42,31.62,31.87,31.27,31.11,30.65,30.47,30.46,30.62,30.17,30.45,29.94,29.73,29.38,29.33,29.33,29.18,29.06,25.92,26.74,26.20,26.93,27.03,26.89,26.71,26.92,26.90,27.08,26.74,27.46,27.06,28.62,31.36,32.15,31.96,31.97,31.03,31.01,30.20,30.71,31.08,31.04,30.51,30.21,29.31,29.81,29.18,29.43,29.07,29.56,26.80,26.61,26.85,26.61,26.86,26.81,26.86,27.13,27.21,26.94,26.84,26.77,27.24,26.68,30.82,31.45,32.56,31.93,31.30,31.13,31.15,31.29,31.83,31.84,31.02,30.16,29.38,29.19,29.56,29.13,29.41,28.83,26.54,26.56,26.58,26.95,26.81,27.11,26.44,27.07,26.81,27.04,26.77,27.05,26.85,26.94,28.29,30.73,31.36,31.84,30.73,30.88,30.73,30.90,31.61,31.92,30.55,30.41,28.76,29.27,29.02,29.64,29.32,29.59,26.79,26.32,26.78,26.93,26.85,26.80,26.95,27.14,26.90,26.59,26.60,26.76,27.19,27.05,27.35,27.16,28.37,28.20,29.84,29.51,31.84,32.69,32.54,31.55,31.03,30.36,29.18,28.63,29.54,28.92,29.18,29.26,26.28,26.71,26.30,27.09,26.60,26.93,26.71,27.05,26.46,27.39,26.90,27.22,26.64,27.33,26.61,27.39,27.15,28.14,28.76,29.57,31.26,32.76,31.79,31.63,30.77,31.15,29.09,29.45,28.78,28.99,29.07,29.83,26.96,26.68,26.70,26.47,26.79,26.53,26.86,26.56,27.08,26.60,26.86,26.50,27.11,26.74,27.29,27.29,27.16,27.50,28.66,28.47,30.35,30.02,30.71,30.36,31.65,31.08,30.13,29.60,29.28,29.08,29.37,29.02,26.45,26.86,26.63,27.07,26.53,27.05,26.85,27.21,26.48,27.28,26.80,27.12,26.54,27.20,26.52,27.19,26.77,27.61,27.91,28.58,28.91,29.42,29.19,30.28,30.07,31.03,30.25,29.85,29.17,29.61,29.30,29.79,26.88,26.71,26.97,26.73,27.01,26.79,26.83,26.99,27.51,26.93,26.96,26.79,27.22,26.88,27.34,26.87,27.20,27.23,27.70,27.45,28.61,28.18,29.45,29.06,29.99,29.77,30.61,29.50,29.72,29.00,29.41,30.15,25.94,27.25,26.77,26.97,26.82,27.24,26.67,27.18,26.75,27.16,27.01,27.37,26.91,27.49,26.81,27.24,26.89,27.97,27.36,28.21,27.73,28.77,28.52,28.88,29.20,30.00,29.95,30.37,29.49,29.48,28.84,29.91

捕獲數據的可視化表示如下：

3 類熱圖像

?

代碼部分中提到的 Github 存儲庫中提供了用于捕獲訓練數據的 Arduino 草圖 (Thermal_camera_data_collection.ino)。為每個類別捕獲了 100 多個樣本。收集的數據已分為訓練 (60%)、驗證 (20%) 和測試 (20%) 數據集。由于數據是從已經校準的紅外溫度相機 (MLX90640) 收集的，并且它們已經在指定范圍內，因此我們可以按原樣使用原始數據進行訓練和推理。

模型架構

我們可以將輸入數據視為 24x36 像素的圖像。卷積神經網絡是適合識別圖像和時間序列數據中的模式的最佳選擇之一。前幾層是帶有少量其他正則化層的 2D 卷積神經網絡。最后一層是具有softmax激活的全連接密集層，它輸出所有3個類別的概率。該模型的總結如下。

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 24, 32, 8)         80        
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 24, 32, 8)         584       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 12, 16, 8)         0         
_________________________________________________________________
dropout (Dropout)            (None, 12, 16, 8)         0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 12, 16, 8)         584       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 8, 8)           0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 6, 8, 8)           0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 6, 8, 16)          1168      
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 3, 4, 16)          0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 3, 4, 16)          0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 3, 4, 16)          2320      
_________________________________________________________________
flatten (Flatten)            (None, 192)               0         
_________________________________________________________________
dense (Dense)                (None, 64)                12352     
_________________________________________________________________
dropout_3 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 32)                2080      
_________________________________________________________________
dropout_4 (Dropout)          (None, 32)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 3)                 99        
=================================================================
Total params: 19,267
Trainable params: 19,267
Non-trainable params: 0

模型訓練和評估

該模型的訓練是在帶有 Linux 和 eGPU (NVIDIA GTX 1080Ti) 的 Intel NUC 上完成的。盡管在 CPU 上訓練只需要幾分鐘，但在測試不同的架構和超參數時，開發過程變得非常緩慢。帶有 Keras API 的 TensorFlow 2.1 用于模型創建和訓練過程。我創建了一個 Jupyter notebook 用于數據處理、訓練和最終模型轉換。所有代碼都可以在代碼部分提到的 Github 存儲庫中找到。訓練準確率為 99%，測試數據的評估準確率為 92.86%，可以通過更多的訓練數據集和模型超參數調整進一步提高。

在設備上進行推理

將創建的模型轉換為 TensorFlow Lite 模型，并將轉換后的模型轉換為 C 數組文件，以便與推理代碼一起部署。TensorFlow Lite Micro SDK 用于在設備上運行推理。我創建了一個 Arduino 草圖（Github 存儲庫中提供 Hot_flash_detector.ino），用于推斷和顯示結果。微控制器以 500 毫秒的間隔連續接收來自熱像儀傳感器的樣本，并根據最近 20 個平均最高溫度讀數檢測被識別人員的溫度突然變化。

共同利益和成本的用例

這是一種易于使用的低功耗設備，可以使用電池運行數周。它可以安全地用于白天或晚上面臨潮熱問題的女性。該設備的成本也很低。最終工作產品（熱像儀 + Atom 矩陣）的總成本遠低于 50 美元，如果批量生產，還可以進一步降低。

改進范圍

該設備可用于通過分析熱圖像時間序列數據來檢測睡眠問題，并可以觸發音樂系統播放一些舒緩的音樂或控制智能照明系統。此外，一些分析數據可以本地保存在 Atom Matrix 的 SPI 閃存中，并且可以使用手機應用程序通過 BLE 進行同步以進行進一步分析。