Keep Sensors in Check: Disentangling Country-Level Generalization Issues in Mobile Sensor-Based Models with Diversity Scores
| Type of publication: | Conference paper |
| Citation: | Nanchen_AIES_2023 |
| Publication status: | Accepted |
| Booktitle: | Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society |
| Year: | 2023 |
| Month: | May |
| Abstract: | It has been established that machine learning models trained with passive sensor data from mobile devices can be used to perform various inferences pertaining to activity recognition, context awareness, and health and well-being. Prior work has focused on improving inference performance through the use of multimodal sensors (inertial, GPS, proximity, app usage, etc.) or improved machine learning. In this context, a few studies shed light on critical issues relating to the poor cross-country generalization of models due to distributional shifts across countries. However, these studies have largely relied on inference performance as a means of studying generalization issues, failing to investigate whether the root cause of the problem is linked to specific sensor modalities (independent variables) or the target attribute (dependent variable). In this paper, we study this issue in complex activities of daily living (ADL) inference task, involving 12 classes, by using a multimodal, multi-country dataset collected from 689 participants across eight countries. We first show that the ‘country of origin’ of data is reasonably captured by sensors and can be inferred from each modality separately, with an average accuracy of 65%. Then we propose two diversity scores (DS) that measure how a country differentiates itself from other countries w.r.t sensor modalities or activities. Using these diversity scores, we observed that both individual sensor modalities and activities have the ability to differentiate countries. However, while many activities capture country differences, only the `App usage' and `Location' sensors can do so. By dissecting country-level diversity across dependent and independent variables, this study provides a comprehensive framework for better understanding model generalization issues across countries and the country-level diversity of sensing modalities. |
| Keywords: | |
| Projects |
Idiap WeNet |
| Authors | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
|
|
Notes
|
|
|
|
|
