Three-dimensional surface information in pretrained EO embeddings

Suggested by: Lisa Mandl

Background: Embeddings are derived from images taken from above, but these images contain indirect information about height, such as shadows, texture or radar backscatter from building walls. It is unclear whether embeddings keep this information or mainly encode the land cover type that usually goes with a certain height. AlphaEarth Foundations combines many data sources, including spaceborne lidar, and uses spatial context. TESSERA works pixel by pixel on Sentinel-1 and Sentinel-2 time series.

Short description: The thesis tests how much height information AlphaEarth Foundations and TESSERA embeddings contain, using airborne laser scanning (ALS) data (where available) as reference. Simple models are trained on the embeddings to predict object heights, with classical Sentinel-1 and Sentinel-2 features as a baseline. The key question is whether embeddings capture height differences within a class (e.g., low vs. high buildings) or only between classes. Study area and objects (e.g. buildings, vegetation) are flexible.

Building height map, here for Berlin. How much of height information is stored in geospatial embeddings? Data source: https://zenodo.org/records/4066295#.X4QRNmgzYmI

Suggested reading

Brown, C. F., Kazmierski, M. R., Pasquarella, V. J., Rucklidge, W. J., Samsikova, M., Zhang, C., et al. (2025). AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label data. arXiv:2507.22291.

Feng, Z., Atzberger, C., Jaffer, S., Knezevic, J., Sormunen, S., Young, R., et al. (2025). TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis. arXiv:2506.20380. https://doi.org/10.48550/arXiv.2507.22291

Frantz, D., Schug, F., Okujeni, A., Navacchi, C., Wagner, W., van der Linden, S., & Hostert, P. (2021). National-scale mapping of building height using Sentinel-1 and Sentinel-2 time series. Remote Sensing of Environment, 252, 112128. https://doi.org/10.1016/j.rse.2020.112128

Start: As soon as possible

Prerequisites/qualification

• Remote Sensing
• Good Python skills + basic machine learning
• Interest in the topic