Automated monitoring and alerting system based on EO image time series

Supervisors

Martin Sudmanns, Dirk Tiede

Background

Temporally high-frequency observations from Copernicus Earth observation (EO) satellites allow detecting and monitoring changes on the Earth’s surface. Changes may include short-term events (e.g. deforestation, flooding) or long-term trends and transitions (e.g. climate-change-induced vegetation changes). Earth observation data cubes are state-of-the-art infrastructure backbones to easier investigate the temporal dimension at scale. It is then possible to detect changes and produce information about types of changes retrospectively using existing time series data in the archives. The “live” monitoring based on continuously updated, new data (e.g. every few days for Sentinel-2 satellite images) in existing approaches are either limited to a specific application in the EO domain (e.g., for deforestation) or developed outside the EO domain and not yet applied and used in combination with EO data / EO data cubes (e.g. Grafana for monitoring IT systems).

Expected from the master thesis is an investigation of existing classifications of EO image time series changes and approaches to monitoring (natural) resources using EO data. Further, a generic method should be developed and (prototypically) implemented as a monitoring and alerting system based on frequently updated EO data cubes. This master thesis will be embedded into the overarching goal of building a semantic EO data cube infrastructure, which is developed at Z_GIS (https://sen2cube.at), and access to these data cubes will be provided.

Example dashboard based on Grafana for monitoring IT resources, including options to configure alerts for increasing, decreasing, or missing values.

Suggested reading

Hermosilla, T., Wulder, M. A., White, J. C., Coops, N. C., & Hobart, G. W. (2018). Disturbance-informed annual land cover classification maps of Canada's forested ecosystems for a 29-year landsat time series. Canadian Journal of Remote Sensing, 44(1), 67-87. https://www.tandfonline.com/doi/full/10.1080/07038992.2018.1437719

Kennedy, R., et al. Bringing an ecological view of change to Landsat‐based remote sensing. Frontiers in Ecology and the Environment 12.6 (2014): 339-346. https://esajournals.onlinelibrary.wiley.com/doi/full/10.1890/130066

Augustin, H., Sudmanns, M., Tiede, D., Lang, S., & Baraldi, A. (2019). Semantic Earth observation data cubes. Data, 4(3), 102. https://www.mdpi.com/2306-5729/4/3/102

Tiede, D., Baraldi, A., Sudmanns, M., Belgiu, M., & Lang, S. (2017). Architecture and prototypical implementation of a semantic querying system for big Earth observation image bases. European journal of remote sensing, 50(1), 452-463. https://www.tandfonline.com/doi/abs/10.1080/22797254.2017.1357432

Related projects

https://sen2cube.at

https://sims.sen2cube.at

Prerequisites/qualification

Remote sensing
Programming and databases

 

Modeling risk for VRUs at intersections

 Suggested by: Martin Loidl, Bernd Lackner

Short description: While the total number of fatal crashes has been decreasing over the past decades, the trend cannot be observed for vulnerable road users (VRUs). Approximately half of all crashes with bicyclists involved occur at intersections. The design of the intersection, the behaviour of other road users, and environmental factors (weather, light, icy conditions etc.) play a central role for the overall risk. Commonly used digital road networks (graph with nodes and edges) represent intersections insufficiently. Thus, crash occurrences at intersections can hardly be put into a spatial context. This is why spatial risk models for VRUs are mainly limited to road links.

Research for this master thesis should address one or more of the following topics (but is not limited to):

• Design of a suitable data model for representing intersections.
• Linking this data model with commonly used road graphs (OpenStreetMap, GIP, ...).
  
• Concept for a spatial risk model for VRUs at intersections.
• Demonstrator implementation of risk model for pedestrians and/or bicyclists into a GIS.
• Validation of risk model.
  

References, suggested reading:

• SCHEPERS, P., HAGENZIEKER, M., METHORST, R., VAN WEE, B. & WEGMAN, F. 2014. A conceptual framework for road safety and mobility applied to cycling safety. Accident Analysis & Prevention, 62, 331-340.
• STOKER, P., GARFINKEL-CASTRO, A., KHAYESI, M., ODERO, W., MWANGI, M. N., PEDEN, M. & EWING, R. 2015. Pedestrian Safety and the Built Environment: A Review of the Risk Factors. Journal of Planning Literature, 30, 377-392.
• LOIDL, M. 2020. Digital abstrahiert − räumliche Daten für die Mobilitätsforschung und Verkehrsplanung. In: ZAGEL, B. & LOIDL, M. (eds.) Geo-IT in Mobilität und Verkehr. Berlin und Offenbach: Wichmann Verlag / VDE. (access to PDF upon request)
  

Related to projects: Bike2CAV, SINUS

Start/finish: anytime

Prerequisites/qualifications: Profound skills in data modelling and processing. Scripting, database and spatial analytics (network analysis) skills.
Depending on interest and availability, this master thesis project could be linked to a study assistent position.

 

Spatial transferability of deep learning models for dwelling extraction in refugee camps and IDPs

Potential supervisory team: Stefan Lang, Dirk Tiede, Lorenz Wendt, Getachew Gella

Globally there is a profound number of the global population displaced from their homes because of natural and anthropogenic reasons. These forcibly displaced segment of the global population resides either in IDP sites or refugee camps if they cross international borders. Dwelling extraction and camp monitoring for a specific single camp can be efficiently done by using deep learning models and high-resolution satellite images and annotated labels to train the model. The biggest problem is when new IDPs/refugee camps are happening and rapid information retrieval is needed for emergency response for humanitarian response. The proposed work intends to leverage the learned skills of the model trained in IDP/refugee camps to extract features in completely unseen images taken from other new IDP/refugee camps. One of the constraints which limit this learning process is that dwelling characteristics in different refugee camps would be completely different which is governed by the medium of the material used to construct the houses, its contrast with the surrounding information, sizes of houses, and spacing of houses (See Fig. 1). A model trained in one site is said to be universally capable of extracting features if and only if it is spatially transferable and able to work very well in refugee camps that are located in other geographic locations. Therefore, the objective of the proposed work will be firstly assessing the spatial transferability of the instance segmentation model and designing implementation strategies that can improve the spatial transferability of the model.

Figure 1: Dwelling type and structures in three different refugee camps

Learning from grey data: implication of input quality on deep learning-based dwelling extraction in IDPs/Refugee camps

Potential supervisory team: Stefan Lang, Dirk Tiede, Lorenz Wendt, Getachew Gella

Deep learning-based dwelling extraction in internally displaced population sites (IDPs) and refugee camps, satellite images could be obtained with different bit depth, processing labels (calibration, atmospheric correction, and further enhancement). With the same token, labeled/annotated data may also be obtained from crowd sources or in-house generated databases with a different label of quality where there might be class mislabelling, imprecise dwelling boundaries (Fig 1), completely unlabelled objects, and others. Therefore, the main theme of the proposed topic will be investigating the implication of investing intensive reprocessing phases on the image side or simply using a co-registered image on the performance of dwelling extraction. Undertaking intensive image pre-processing steps and the generation of the very precise respective object labels for model training and testing demands a profound amount of time. Especially when it comes to operational settings in emergency response for humanitarian response, it could be a pressing challenge for rapid mapping and information retrieval. Therefore, the proposed study will focus on two issues. Firstly, a thorough investigation will be done on trade-offs of using a clean but small amount of labeled sample data and a large number of grey samples with less precise annotation quality for dwelling extraction using deep learning models. Second, a substantial effort would be invested in the creation of robust deep learning strategies that could leverage from large grey data in a learning phase

Figure 1: Satellite image of dwelling features(A) and image with annotated labels with shifted object boundary and two mislabeled objects

Dealing with imbalanced data of forcibly displaced people’s dwellings

Suggested/supervised by: Yunya Gao, Stefan Lang

Due to human armed conflicts, human rights violations, persecution, or environmental degradation, numerous people are being forced to flee their homes globally. They are called refugees if they have crossed international borders, or internally displaced persons (IDPs) in the case they are on the run within their home countries. Refugee/IDP dwellings are commonly used as temporary shelters for these displaced people. Detailed information on refugee/IDP dwelling infrastructure, the population in need and their spatial distribution is important for planning humanitarian actions (Sprohnle, Fuchs, & Aravena Pelizari, 2017). However, during a crisis, such critical information is usually hard to access by fieldwork (Witmer, 2015). Therefore, detecting refugee/IDP camps through remote sensing (RS) techniques has attracted a lot of attention.

Very high spatial resolution optical (VHSRO) satellite imagery is considered as the major source of information to identify the number and types of structures (Ghorbanzadeh, Tiede, Wendt, Sudmanns, & Lang, 2021). Visual interpretation of VHSRO imagery is recognized to produce accurate refugee/IDP camp mapping results (Witmer, 2015). However, it is thought to be highly reliant on context knowledge, time-consuming and labour intensive (Witmer, 2015). Recently, deep learning (DL) methods have been proved to outperform many other automatic processing methods in many remote sensing domains. However, DL methods usually require a large amount of data and high computational cost (Zhu et al., 2017). In the past few decades, ZGIS has assisted Doctors without borders (Médecins Sans Frontières, MSF) in collecting a large amount of data of refugee/IDP camps together with VHSRO imagery for many countries. Therefore, DL methods have high potentials in detecting refugee/IDP dwellings. However, by looking through the collected data, there are usually multiple classes of dwellings. The amount of data for each class is imbalanced. After some initial experiments, it is found out that the accuracy of minority classes (e.g. Tukul) is usually much lower than majority classes (e.g. bright dwellings). It is still unknown how to deal with imbalanced data issues in refugee/IDP dwelling detection by using DL methods. Improving the accuracy of prediction results of minority classes may be valuable for planning humanitarian actions.

Figure 1. Three types of refugee/IDP dwellings represented from very high spatial resolution satellite imagery

Reference:

• Ghorbanzadeh, O., Tiede, D., Wendt, L., Sudmanns, M., & Lang, S. (2021). Transferable instance segmentation of dwellings in a refugee camp - integrating CNN and OBIA. European Journal of Remote Sensing, 54(sup1), 127–140. https://doi.org/10.1080/22797254.2020.1759456
• Sprohnle, K., Fuchs, E. M., & Aravena Pelizari, P. (2017). Object-Based Analysis and Fusion of Optical and SAR Satellite Data for Dwelling Detection in Refugee Camps. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(5), 1780–1791. https://doi.org/10.1109/JSTARS.2017.2664982
• Witmer, F. D. W. (2015). Remote sensing of violent conflict: eyes from above. International Journal of Remote Sensing, 36(9), 2326–2352. https://doi.org/10.1080/01431161.2015.1035412
• Zhu, X. X., Tuia, D., Mou, L., Xia, G. S., Zhang, L., Xu, F., & Fraundorfer, F. (2017). Deep learning in remote sensing: a review. ArXiv, (December). https://doi.org/10.1109/MGRS.2017.2762307

Assessing the potential of freely available Earth Observation (EO) data as malaria risk indicators in sub-Saharan Africa

Suggested/supervised by: Linda Menk

Background:

Despite huge achievements in the prevention and control of malaria, still more than 200 million humans globally fall ill with it every year. About 95% of the cases and related deaths happen in sub-Saharan Africa. The population group at highest risk are children under the age of 5.

Malaria is transmitted by certain mosquito species which are dependent on certain habitat preconditions. For example, Vegetation health (VH) was found to be a good indicator of annual mosquito activity and related spikes in malaria outbreaks.

Source: Malaria Atlas Project (https://malariaatlas.org/)

The European flagship programme on Earth Observation Copernicus offers a growing variety of global datasets which are derived from satellite imagery, such as the Vegetation Condition Index and the Vegetation Productivity Index. The departure point for this master thesis could be the exploration of the global datasets offered by Copernicus (and others) and their potential to function as malaria risk indicators. After exploring which datasets could be put to use and why, the data should be retrieved, integrated and visualized for a specific region of interest in Africa. The master thesis will be part of a malaria case study which is currently conducted together with Doctors without borders/Médecins sans Frontières (MSF), as part of the Christian Doppler Laboratory “GEOHUM”. The results will help MSF to focus their malaria prevention activities to areas which face the highest risk.

Further reading: 

• https://land.copernicus.eu/global/themes/vegetation
• https://link.springer.com/chapter/10.1007/978-3-030-46020-4_6
• https://www.who.int/teams/global-malaria-programme/reports/world-malaria-report-2020/

Developing EO routines for humanitarian situational awareness in Cabo Delgado Province, Mozambique

Suggested/supervised by: Lorenz Wendt

Background:

In the northern province of Mozambique, Cabo Delgado, Islamic extremist armed groups started aggressive attacks on civilians in October 2017. These attacks have continued in 2021, causing insecurity and displacement within the area, food shortages, and further suffering by the backlash of government forces. An estimated 713,000 people have been internally displaced, mostly from the disputed coastal areas towards more inland regions according to estimates by UNHCR and IOM.

The conflict zone is partly considered a “no access /hard to reach area”, making it difficult to get a comprehensive overview of the situation for humanitarian actors like MSF or UN organisations. The task is therefore to leverage EO data to map the effect of the conflict on people, villages, transport infrastructure and agricultural production. Multitemporal optical imagery from Sentinel-2 and potentially Planetscope shall be used to identify abandoned villages and roads by mapping overgrowth. Newly formed settlements in the safer regions surrounding the conflict zone may be detected by bushland or forest clearing. A reduction of active cropland might indicate displacement of people and/or food insecurity. These analyses can be carried out in Google Earth Engine or other cloud processing platforms, following strategies employed also by the World Food Programme.

In addition, VHR images might be analyzed to map dwellings (tents, huts, etc) of IDPs in selected hotspot locations, for example Pemba City (MOZ). Given the breadth of EO and GI data available, the exact scope, dataset and methodology will be discussed with the MSc candidate individually.

Further reading:

• https://www.acaps.org/country/mozambique/crisis/violent-insurgency-in-cabo-delgado
• https://reliefweb.int/report/mozambique/urgent-needs-mozambique-cabo-delgado-situation-7-may-2021
• https://displacement.iom.int/reports/iom-dtm-baseline-assessment-report-round-11-march-2021
• https://www.wfp.org/publications/wfp-mali-satellite-imagery