Project 1: Tina

Squad: Intelligent Agents

Introduction

Everyone experiences emotions, everyday of their lives. Sometimes it is clear where these emotions come from, and how to deal with these emotions. Other moments, a person might have a hard time understanding where emotions come from, how to process the emotion and what to do with it. One’s ability to perceive, use, process and deal with emotions, is also called Emotional Intelligence (Hulac D.M. et all., 2020).

Emotional intelligence (EI) is a topic that is getting more acknowledged in professional careers than ever before, the EQ is even considerably more valuable than the old IQ system (Salovey P. et all., 2005). EI can have an enormous impact on your daily life, socially, professionally as well as privately, and is something that can be trained. By training someone’s emotional intelligence, both emotional and behavioral problems can be reduced. This results in better functioning throughout the day and improved social skills and learning ability (Salovey P. et all., 1990).

Product description

The goal of this project was to positively influence people’s feelings through their environment while also improving their Emotional Intelligence. By analyzing hormones in a user’s sweat using the Tina hormone meter, Tina is able to determine your current emotion. The information will then be processed by the agent and linked to an emotion. Based on this emotion, trends, and other data such as time and calendar appointments, the agent will influence the environment that will have a positive effect on the user.

In addition, Tina helps to understand emotions by reflecting together with the user. This makes the user more aware of the emotions he/she is experiencing, and will make them more emotionally intelligent. 

Extended product description

The hormone meter

The hormone meter would work as follows: Sweat gets on the sensor of the hormone meter. When a certain hormone is present, that hormone will bind to the antibody of that hormone that is present on the MoS2 nanosheet. With this bond, the wavelength of the nanosheet’s light reflection will change. The light coming from the LED will then arrive at the spectrometer with a different wavelength. This makes it possible to measure whether and how much of
each hormone is present. The data from the spectrometer will be stored in the data logger, which will share the data with the agent via Bluetooth. Through the combination of hormone percentages, Tina can then estimate the current mood of the user.

The use of sweat makes it possible to collect hormone data in a non-invasive, user-friendly way. User-friendly because the user is not restricted in movement, which could be the case with the use of blood. Furthermore, the user does not have to perform any action himself to be able to take a hormone measurement, which would be necessary when using saliva. The user can also easily remove the device by means of the plasters. And it is possible to wear the meter in a place on the body that the user finds most comfortable.

The necessary resources are currently available to realize a hormone meter with this function, and accuracy of measurements. For example, a portable cortisol meter has already been developed that can measure hormone levels very accurately using the same MoS2 nanosheets (Kinnamon, 2017). However, it is not yet fully possible to make the hormone meter small enough with this number of measurable hormones to be burden-free on someone’s body. It would be possible to measure one specific molecule with a slightly larger device (the size of a watch).

The algorithm

For Tina to work, there are two important algorithms necessary: First of all, the recognition of hormones, and second the corresponding action based on this emotion. Around this, voice recognition is necessary. For this we decided to use Mycroft AI, an open source and private assistant. This software runs completely offline (FAQ – Mycroft, 2021), thus ensuring privacy.

The normal hormone levels can be very different for every person (Lauretta et al., 2018). Consequently, it is hard to predict the normal levels of a user. That is why this part of the algorithm will learn from the user and adjust to them. Based on age and sex certain predictions of what the normal levels will be can be made (Lauretta et al., 2018). With these predictions the user can get a preliminary dataset for normal values, but over time their own measurements can be used to determine the emotion.

A calibration period is therefor necessary to customize the normal values to the
user. A preliminary algorithm was created which creates a dataset with random values based on normal hormone levels in adult males. It then compares a subset of the dataset to the entire dataset so it can see if the current levels differ from that person’s normal levels. To determine this, certain parameters were used to determine emotion. These parameters can be changed to fit the person’s preferences. This will be done by asking the user for confirmation of their emotion during the calibration period. That way if it is correct, the parameters can stay the same. If an emotion is detected, the algorithm will see over time if this continues, to eliminate uncertainties in the measurements. The algorithm can also be connected with the user’s calendar, to determine if certain upcoming appointments can cause certain emotions. The algorithm will also look at recurring patterns. For example, if there is a yearly recurring stressful period, Tina can make recommendations based on that for the next year. If there is enough reason to make an action, it will move on to the second algorithm.

The second algorithm is to determine the course of action. The preferred course of action with certain emotions can be different for everyone, so Tina can learn from the user which actions are good. At the beginning certain actions are standard, for example, when someone is depressed it might help to drink coffee (Lucas, M. et al., 2011; Ruusunen, A. et al., 2010).
There are several types of actions Tina can do:

  • Clearly visible: making coffee, running bath, changing lights etc, see also appendix 10
  • Less visible: changing schedule,
  • Recommendations: sleep more, eat this, meditate etc
  • Moments of reflection: To increase the user’s emotional intelligence, Tina will ask the user to reflect on their day. This will happen at a time the user will like, by trial and error Tina will determine which time is convenient for the user. By asking feedback on the moment or by direct feedback from the user, Tina can determine what moment is good. The user can also instigate it directly by talking to Tina.

Tina can learn whether an action was useful by looking at the hormone levels, if they go back to normal, or at least improve a bit, the action was useful. In the beginning it can also ask the user if they think that was a correct action, but over time Tina can become more autonomous. Of course the user can alway override an action if they do not agree with Tina. Some of these actions are not possible for some users as they require changed products, for example a coffee machine that automatically makes coffee or an automatic bath. In this case these options won’t be tried by Tina.

The speaker

The Tina speaker serves as an interaction device for the user. It is designed to be as user-friendly as possible. The speaker has a soft exterior which is pleasant to touch. The speaker invites the user to fidget which is a stress and anxiety reducing interaction (Hulac, 2020). It really enables the user to feel a sense of connection to Tina and allows you to improve your Emotional Intelligence at peace of mind.

Both the speaker and hormone meter are color customizable to the user’s preferences, this way Tina will seamlessly connect to the user’s aesthetics

References

  • FAQ | Mycroft AI. (2021). https://mycroft-ai.gitbook.io/docs/about-mycroft-ai/faq
  • Hulac, D., & Aspiranti, K. B. (n.d.). A Multisite Study of the Effect of Fidget Spinners on Academic Performance. https://doi.org/10.1007/s40688-020-00292-y
  • Kinnamon, D. D., Morales, A., Bowen, D. J., Burke, W., Hershberger, R. E., Morales, A., Bowen, D. J., Gastier-Foster, J. M., Nickerson, D. A., Dorschner, M. O., Haas, G., Abraham, W., Binkley, P., Hasan, A., Host, J., Lampert, B., Smith, S., Huggins, G., Denofrio, D., … Hindorff, L. (2017). Toward Genetics-Driven Early Intervention in Dilated Cardiomyopathy: Design and Implementation of the DCM Precision Medicine Study. Circulation. Cardiovascular Genetics, 10(6). https://doi.org/10.1161/CIRCGENETICS.117.001826
  • Lauretta, R., Sansone, M., Sansone, A., Romanelli, F., & Appetecchia, M. (2018). Gender in Endocrine Diseases: Role of Sex Gonadal Hormones. International Journal of Endocrinology, 2018(1), 4847376. https://doi.org/10.1155/2018/4847376
  • Lucas, M., Mirzaei, F., Pan, A., Okereke, O. I., Willett, W. C., O’Reilly, É. J., Koenen, K., & Ascherio, A. (2011). Coffee, Caffeine, and Risk of Depression Among Women. Archives of Internal Medicine, 171(17), 1571–1578. https://doi.org/10.1001/ARCHINTERNMED.2011.393
  • Ruusunen, A., Lehto, S. M., Tolmunen, T., Mursu, J., Kaplan, G. A., & Voutilainen, S. (2010). Coffee, tea and caffeine intake and the risk of severe depression in middle-aged Finnish men: the Kuopio Ischaemic Heart Disease Risk Factor Study. Public Health Nutrition, 13(8), 1215–1220. https://doi.org/10.1017/S1368980010000509