Generate Godot Resources From CSV

I'm going a bit more technical than usual with this post as I'm going to be sharing some of this information with others anyways. Two birds with one stone and all of that...

Resources are data containers. They don't do anything on their own: instead, nodes use the data contained in resources.
- Godot Documentation

I won't be reviewing Resources themselves, so if you need more info, please read more from the Godot documentation.

One of the most obvious uses of custom Resources in my game Cursor Incremental is for the upgrades. There will likely be dozens of upgrades, each with 1-100(?) levels, with each level requiring different types and amounts of resources.

This is an obvious use case for Resources. Here's what my Upgrade resource looks like

class_name Upgrade extends Resource

@export var icon:Texture2D
@export var id: StringName
@export var target_entity: StringName
@export var name: String
@export var enabled: bool = true
@export_multiline var description: String
var current_level:int = 1
##Upgrade id
@export var prereq1_id:StringName
##[level, upgrade_level]
@export var upgrade_levels:Dictionary[int, UpgradeLevel]

If you look at the last line, there's actually another resource type nested in this resource.

class_name UpgradeLevel extends Resource

@export var upgrade_id:StringName
@export var level:int
@export var costs:Dictionary[InventoryItem, int]

And there's even more nesting, I have the InventoryItem resource being used as a key.

extends Resource
class_name InventoryItem

@export var id: String
@export var name: String
@export_multiline var description: String
@export var sprite_texture: Texture2D

Problems Managing Resources in the Godot Editor

Once I started trying to edit my upgrade resources I noticed the nested properties were... problematic.

I tried the ResourceTables plugin which is actually quite nice and allows editing resources in a CSV like environment, but it also doesn't handle collections of nested Resources.

I wanted to manage the data in something more human readable, like CSV. There were probably other alternatives out there, like JSON, or YAML, but I often find a simple CSV or spreadsheet much easier to manipulate, so I decided to stick with that.

Editing CSVs (Almost) Straight From the Godot Editor

I wanted my CSV data to stay local to my godot project because it would allow versioning of the changes via Git, plus it would cut down on import/export to external tools like Excel or Google Sheets. I also dislike paying for tools I don't fully use. Modern CSV has filled the gap for me. Although there's a paid version I get by on the free version. In Godot I have setup Modern CSV as the external program to use for all my csv files.

Now with barely leaving my editor I can edit my upgrade data. The one compromise I had to make by using CSV is that I need to maintain two separate tables, one for the Upgrade, and one for the UpgradeLevels. This felt nicer to me than having one sheet and duplicating the Upgrade data for every UpgradeLevel row, but it also meant mapping them together in code.

Onto The Code!

I implemented the following Resource generation features

• Parse a CSV into multiple resources
• Write those resources to files
• Re-running the generator merges only the CSV controlled data, leaving other editor controlled data, such as images, untouched.

I wrote two custom utility classes that are required for parsing CSV and writing Resources to files.

Generating the Resources From the Godot Editor

And here's the coup de grâce, putting it all together. The code is defined as a @tool and extends EditorScript, meaning it can be run from the Godot Editor with File -> Run or CTRL+SHIFT+X. When run it reads the two provided CSVs, and runs code specific to my game's Upgrade resources to merge the two sheets into a single list of Upgrades resources, each with a collection of UpgradeLevels. This way I no longer need to manage UpgradeLevels directly in my code.

@tool
## Editor tool script for importing and managing upgrade data from CSV files.
##
## This script loads upgrade definitions and upgrade levels from CSV files,
## validates their relationships, merges them into existing resource files,
## and saves or deletes `.tres` resources accordingly. It also updates the
## resource reference script for editor integration.
extends EditorScript

var inventory_manager:InventoryManager = InventoryManager.new()
var upgrade_manager:UpgradeManager = UpgradeManager.new()

const UPGRADE_CSV_PATH  = "res://scenes/managers/upgrades/csv/upgrades.csv"
const UPGRADE_LEVEL_CSV_PATH  = "res://scenes/managers/upgrades/csv/upgrade_levels.csv"
const SAVE_DIR = "res://scenes/managers/upgrades/data/"

const UPGRADE_ID_PARAM_NAME:String = "id"

## Entry point when the script is run from the Godot Editor.
##
## Loads CSV data into resource instances, validates it, merges with existing resources,
## saves new or updated resources, deletes obsolete ones, and refreshes the editor's file system.
func _run():

    # Parse and validate CSVs
    var upgrades:Array[Upgrade]
    upgrades.assign(CSV.parse_csv_to_resource_list(Upgrade, UPGRADE_CSV_PATH))
    assert( ResourceUtil.has_duplicate_ids(upgrades, UPGRADE_ID_PARAM_NAME) == false, "Duplicate upgrade id detected, ending resource generation")

    var upgrade_headers := CSV.get_headers(UPGRADE_CSV_PATH)
    var upgrade_levels:Array[UpgradeLevel]
    upgrade_levels.assign(CSV.parse_csv_to_resource_list(UpgradeLevel, UPGRADE_LEVEL_CSV_PATH, map_csv_values_to_upgrade_level) as Array[UpgradeLevel])

    #Create id based dictionary of resources
    var upgrades_by_id:Dictionary[StringName, Upgrade] = {}
    upgrades_by_id.assign(ResourceUtil.resources_to_id_dictionary(upgrades, UPGRADE_ID_PARAM_NAME))

    #Add levels to matching upgrades
    for upgrade_level in upgrade_levels:
        var upgrade = upgrades_by_id.get(upgrade_level.upgrade_id)
        assert(upgrade != null, "Upgrade level with no associated upgrade_id: " + str(upgrade_level.upgrade_id))
        assert(!upgrade.upgrade_levels.has(upgrade_level.level),  "Duplicate upgrade level: " + str(upgrade_level.upgrade_id) + ":" + str(upgrade_level.level))
        assert(upgrade_level.level >= 1, "Invalid level" + str(upgrade_level.upgrade_id) + ":" + str(upgrade_level.level))
        upgrade.upgrade_levels[upgrade_level.level] = upgrade_level

    #Validate levels in upgrades
    for upgrade in upgrades_by_id.values():
        var levels:Array[int] =  upgrade.upgrade_levels.keys()
        levels.sort()
        assert(levels.size() == levels.back(), "Invalid number of levels " + str(upgrade.id))


    #Create list of resource properties to merge
    var properties_to_merge = upgrade_headers.duplicate()
    properties_to_merge.append("upgrade_levels")

    #Get all existing ExampleEnemy resources stored in the file system, merge CSV resource into existing ones
    var upgrade_resources_to_update := ResourceUtil.load_resources_in_editor(Upgrade)
    ResourceUtil.merge_resources_arrays(upgrades, upgrade_resources_to_update, UPGRADE_ID_PARAM_NAME, properties_to_merge, true)
    ResourceUtil.save_and_clean_all_resources_to_filesystem(Upgrade, upgrade_resources_to_update, SAVE_DIR, UPGRADE_ID_PARAM_NAME)

    #Create reference script for all Upgrades
    ResourceReferenceGenerator.create_reference_script(Upgrade, UPGRADE_ID_PARAM_NAME)

    # Update editor after file changes
    EditorInterface.get_resource_filesystem().scan()

## Maps CSV values into an `UpgradeLevel` resource instance.[br][br]
##
## This function is used as a custom mapping callable passed into the CSV parser.[br][br]
##
## - `headers`: The header names from the CSV (currently unused, but may be useful for mapping by name).[br]
## - `values`: A row of CSV data as a list of strings.[br]
## - `resource`: The [UpgradeLevel] instance to populate.[br]
func map_csv_values_to_upgrade_level(headers:Array[String], values:Array[Variant], resource:UpgradeLevel) -> void:

    resource.upgrade_id = values[headers.find("upgrade_id")]
    resource.level = int(values[headers.find("level")])
    var wood_cost = int(values[headers.find("wood_cost")])
    if wood_cost > 0:
        resource.set_cost(InventoryItemRef.getr_wood(),wood_cost)
    var stone_cost = int(values[headers.find("stone_cost")])
    if stone_cost > 0:
        resource.set_cost(InventoryItemRef.getr_stone(),stone_cost)

After running the generator I get a complete set of upgrade resource files inside the Godot editor and edit more of the editor-friendly properties such as the icon texture.

Rerunning the generator again will merge in only the properties I have specified, as well as delete resources that no longer exist in the CSV. I can now iterate extremely fast generate tens or hundreds of resources in a matter of minutes.

Learning By Tooling

While a lot of this tooling wasn't really progressing my gameplay, it has taught me a TON about how Godot and GDScript work. I refactored my code 3 or 4 times over already as I learned more and more. I probably cut out half the code as I learned of many existing functions I was unaware of. For example, all of the file path helper functions are part of the String built-in type.

Why Use GDScript and not C#?

While I'm sure some C# libraries may have made some of this work easier, I had arbitrarily decided to use GDScript when I started my Godot journey. After using it for a few months, I think it's also an excellent and easy to understand language for new coders while adding relative flexibility for more experienced programmers.

Additionally being able to stay in a single editor is nice in its own way.

Try It Yourself

I have posted the tools I have talked about here into a public repo for others to play with or use.

GitHub - BigDewlap/dewlap_tools: Collection of scripts that provide helpful functionality or enabled improved workflows

Collection of scripts that provide helpful functionality or enabled improved workflows - BigDewlap/dewlap_tools

GitHubBigDewlap

A simplified example can be found here. Note the simplified example also includes a more advanced topic on static access to generated resources that I will post about next week.

Keep an eye out for some more technical blog posts coming up as I have built some other useful tools for dealing with resources.