Elasticsearch/Kibana: 3 ways to add missing fields during runtime
Generate fields during runtime in visualizations with “runtime fields”, “lens formulas” or “scripted fields”. But beware: they are coming with a price. If you miss fields and query them regularly, better create them during indexing.
Solution 1: Create runtime_mappings
To add a runtime field, go to Stack Management -> Index Patterns -> [your index pattern] -> Add field
Now add a runtime field name and set the right data type. Add a custom label, this will be the name that will be visible in your lens visualization. And then emit a value that is computed from existing fields or by a Painless Script.
Solution 2: add a lens formula
I covered that already in Kibana Lens Formulas. Just as a quick recap: while you are creating a visualization in Lens, add a formula in the metrics section:
Solution 3: add a scripted field
This is deprecated! But for completeness and for those who run old versions of Kibana/Elasticsearch, I will quickly show how a scripted field can be added. Go again to To add a runtime field, go to Stack Management -> Index Patterns -> [your index pattern] and change to the tab “Scripted fields”. Then add a scripted field like below:
Conclusion
Fields that are generated during runtime are considered expensive. Be careful and use them wisely. If you need them regularly, consider adding these fields with a pipeline and a script for new data and the _update_by_query endpoint using the same script for adding these fields for the existing data.
Additional resources
https://www.elastic.co/guide/en/kibana/current/lens.html
https://www.elastic.co/guide/en/kibana/current/managing-index-patterns.html#runtime-fields
