this post was submitted on 22 May 2025

18 points (95.0% liked)

Programming

20279 readers

503 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev

founded 2 years ago

MODERATORS

snowe@programming.dev

Ategon@programming.dev

MaungaHikoi@lemmy.nz

UlrikHD@programming.dev

How do I keep all the data I need in one single place for my website? (feddit.it)

submitted 12 hours ago by dontblink@feddit.it to c/programming@programming.dev

6 comments fedilink hide all child comments

I need to create a website that holds some events data as well as other content, it will also maybe need to grab some data from APIs.

Since I'm skilled with Hugo (static site generator) I thought I could use that but it's turning out it's a total mess actually.

In Hugo I can have contacts (like events organizers) as taxonomy, but that is a different format (yaml) than CSV or vcard, and it's also static, meaning that if I edit a contact it will only change in Hugo. So I found myself having to manage contacts in 4 different places, in 4 different ways: Hugo yaml, Thunderbird, google contacts, CSV (from earlier days)... And I will add mailchimp once I'll also add a newsletter. This ensures my contacts are kinda becoming a mess.

Same goes with events, it's okay if I generate events in Hugo, but if I grab events from APIs and then the API content changes I will have to modify it on Hugo as well.

Everything it's turning out to be a total mess essentially and I think I tried to use something simple to build something quite complex, I realized the complexity later.

Now ideally I would like to be able to have my contacts, my newsletter, my content in one single place ato have everything nicely synced and not having to deal with 30 different lists or formats.

What should I do?

I know about the jamstack and headless CMS like ghosts and I was wondering if they could be a good solution, or if I should opt for a full CMS. Obvious solution would be WordPress but I wouldn't really want to mess with all the plugins.

Also I spent quite a but of time in building my templates for the Hugo website and throwing everything away would feel awful, if there's a way to reuse them (?).

I know some JavaScript basics but I would avoid it if possible.

top 6 comments

sorted by: hot top controversial new old

[–] technocrit@lemmy.dbzer0.com 7 points 7 hours ago* (last edited 7 hours ago)

In the olde days people used databases for storing data.

I don't know much about Hugo but, if your data isn't static, then maybe you shouldn't use a static website.

[–] roadrunner_ex@lemmy.ca 2 points 8 hours ago

If you aren't married to Hugo as your solution, I will recommend giving Eleventy.js a look.

It's a static-site generator, but a good amount of flexibility is afforded by virtue of using pure JS to generate view data (which means that you can do any conversions needed, manually or with NPM packages if needed for more proprietary data formats), and it supports a bunch of templating engines too.

[–] Mirror Giraffe@piefed.social 3 points 10 hours ago* (last edited 5 hours ago) (1 children)

One thing that's common when having multiple sources like different data sources, cms and utility services is having a backend for frontend (BFF) which is what your frontend would call and in turn it would gather all your resources and format it the way your frontend expects it. The bff can cache the more static sources while proxying more dynamic ones.

This might be overkill for your project but it's a nice paradigm when you're dealing with multiple sources or sources that might change.

[–] adry@piefed.social 2 points 9 hours ago

Yes, I think the problem description fits for moving from static website to having a database and a web service ( with or without javascript, could be just forms, check htmx.org ). For example, if you know Python you could go and check Django. But there are many other ORMs, all aiming to make working with databases easier. Btw, I am not expanding the ORM acronym because it would only add noise. OP should look into that, and understand why these 'frameworks' are called like that... Good luck!

[–] Kissaki@programming.dev 3 points 11 hours ago (1 children)

Have you considered Hugo Data sources (data subfolder)?

Synchronising a data format in there, synchronising the data through APIs and some tooling, and then generate the Hugo website from that data may simplify the process - make it viable despite still having some variance and complexity.

in 4 different ways: Hugo yaml, Thunderbird, google contacts, CSV (from earlier days)

Are you still using CSV? If it's integratable into one of the other three that could reduce complexity by dropping this fourth case?

What do you want your defining state to be? Or should it synchronise across, and changes must be possible in any of the places?

Thunderbird can use Google contacts and calendar. Some tooling could sync to or from or both your Hugo data folder.

[–] dontblink@feddit.it 1 points 11 hours ago

Thank you for your answer!

Yes I've considered using Hugo data sources, but handling all events in one single data file is not really a good way to manage data because Hugo can't programmatically generate content pages from a single data file sadly.. Also again, even if I make a script able to do this, I don't think you can modify content when already created without handling single posts individually.

I could generate a "list of events" but not individual pages from it and not an RSS feed for posts which I would need for newsletters etc..

The thing with CSV is that I kinda lost track of where the actual updated data is, so I'm keeping that updated too, yeah I know I'm a mess.

All the stuff cited is needed for one single job essentially: contacts, newsletter, events.. Which is gathering self published and externally published events and sending them to a list of chosen emails + some integration with social medias.

I'm not a webdev and I thought I could solve this much more easily, but I think doing this correctly would involve using at least an headless CMS + something that is able to grab data from external APIs + some JS framework for building the frontend.

Or relying on a ready full CMS like Ghost or WordPress + theme and hosting on a VPS, which honestly is what I'm leaning towards..

I want to avoid JS if possible as I had terrible coding experiences with it, I know some Rust but webdev in rust is not really a good option from what I've learned.

What do you think?