Due to the growing importance of the privacy and data protection aspect of smart home and IoT devices, our tests have also evolved further and further in this direction in recent years. Where at the beginning of our IoT certification this topic was already included in the checks but did not yet have any critical relevance, today no product passes our tests without implementing a solid concept for the protection of users’ personal data. More specifically, this means that two critical points must be covered: First, the product handles personal data in a secure manner and protects it from access by any attackers. And secondly, the user is comprehensively informed about the collection, processing and possible sharing of his data via the product’s data privacy statement and can decide for himself on this basis whether its use is acceptable to him. However, we explicitly do not rate the amount of collected data and the associated impact on the user’s privacy.
As with any other test, the first step is to collect all relevant information that we can obtain without actually running the application in question. This is mainly information such as requested permissions, included tracking and conspicuous third-party modules, as well as relevant and interesting areas of the application source code. These can be static URLs, references to communication to certain domains and the like. The goal here is to obtain an initial assessment of the theoretical possibilities and capabilities of the application in question, which can then be substantiated, proven or combined with the results of the practical behavior analysis.
The most extensive and time-consuming area is behavioral analysis. The aim here is to examine the communication behavior of the application as comprehensively as possible and under realistic conditions. A simple observation of the communication of the app in an idle state is of course not sufficient here, since it can be virtually ruled out that all states and thus communication processes of the app are triggered this way. However, in order to be able to guarantee this with a relatively high probability, all app functions must be activated, i.e. simply put, each button must also be pressed at least once. For this purpose, we developed an additional application that automatically, recursively runs through the menu structure of the analyzed application, pressing each interactive UI element, such as buttons, sliders, links, etc., once. In parallel, all communication executed by the application is recorded. The result is a fairly comprehensive overview of the practical communication options of an app – if necessary, we can even associate individual actions on the UI directly with the corresponding triggered communication.
The recorded communication is then searched for interesting and relevant information. This way it is possible to determine the domains to which communication took place, the data and data volume transferred, and whether this data can be linked to information from the static analysis. For example, it is particularly interesting to see whether tracking modules that could be identified statically are also active in practice.