Delay in data availability from real-time to historical download

All of the CSVs and graphs are generated at the same time (same function). Submission comes in --> data stored --> sets alert flag (if in alert) --> processes CSVs and graphs.

I have just had a look in the code and it looks like Dan has set different ages for the different graphs. The CSVs and graphs won't update unless the age is longer than the time stated:
Graph today - 30 seconds
Graph week - 12 hours
Graph month - 3 days
Graph quarter - 7 days
Graph year - 14 days

The CSVs are generated at the same time as the graphs. I think I may understand the angle you are coming from - "Why doesn't the year graph have right up to date counts, as in todays?" If that is a question of yours, the answer is, it doesn't need to be in the year graph. The data is in the other graphs, in more detail, such as graph today and graph week. As each graph reduces in time scale, so from year to quarter, quarter to month, month to week, week to day etc. the data is averaged with less samples, so is more detailed. It's kind of pointless looking at graph year for something that happened a day or three ago as it will be averaged down. A fast spike will be reduced as the counts either side of the spike are calculated with the spike and the number averaged. 

Lets say we take these counts: 21, 23, 25, 22, 135, 75, 33, 23. There is a small spike up to 135, then 75 and back down to background. To average these we add them together, so 21 + 23 + 25 + 22 + 135 + 75 + 33 + 23, then we divide by the samples, 8 in this case, so the sum of the counts is 357 divided by 8 samples give us 44.625. So that fast spike at 135 CPM is averaged down to 44.625, saved in the CSV as such and the graph generated from it. This is assuming that is the kind of averaging method Dan has implemented. So, even if the data was right up to date in the year graph, it wouldn't be an accurate representation of the fast spike anyway.

A couple of things I have wanted to do for a while is completely ditch the current graph generation (and CSVs) and replace them with a new system that dynamically generates the graphs only when called for, as in when someone looks at them - very much as I have on this page: https://www.schmoozie.co.uk/radon_3699/ . And implement a way to interrogate the entire dataset for a station, right back to the last record kept. The idea is to choose a day, or a date range, that will list all of the counts for that period and offer an export mechanism to export to CSV, but only when called for not automatically and maybe only to registered users of radmon, with or without a station. That would save a ton of resources on the server and give all registered users a much better tool for viewing and interrogating data. Another thing that would be handy is a way to users to correct data. Chop out a bit, or change some CPMs. I do it all the time, just throw a piece of uranium or whatever on a counter, forget it's submitting to radmon, then have a big spike in my data, and I may want to get rid of that spike. It would be a handy feature, but Dan and I are reluctant as a user may accidentally delete all of their data, or a user could maliciously include fake CPMs and cry wolf. So unless there were mechanisms to stop that then I doubt it will happen in any detailed fashion. 

And that is all well and good, easy peasy, it's only a bit of code and I have already done it on my website, so just port, right? Not quite that easy. Radmon uses a Joomla front end. It's because of many reasons - security, features, it makes things pretty. Its a bit like the car body over the engine and chassis. But Joomla is a bitch. She is the girlfriend that shags the best, but nags and can't cook. She is the karma for all the shitty things I did when I was a kid. She's a bitch. My relationship with Joomla is somewhat.... like disdain. Joomla is a re-write engine. It takes anything we feed it, the station list, alerts, station page, whatever, it takes those basic not very pretty pages, caches them, parses them and writes out the Joomla page with the content of the page it is fed. It makes things pretty and brings them all under one umbrella. Take the control panel, it's full of form fields. Joomla won't use form fields in a standard manner, it just doesn't read back the form field property when it is submitted. I worked out a way to get them to work, but it is incredibly complicated, complex and requires an 'interface' page writing that will take from one thing and give to the other, so it can read it. It's that complex I really struggle to read back my own code. It's like writing a full PHP page, then chopping it into three, putting one third here, one third there and the last third over there, then writing more to each page to make them talk. It took me about 2 days to write the control panel to run under it's own standalone php page. It took me another three months to integrate it into Joomla, lol. Right now I find sticking needles in my eyes more appealing than coding for Joomla. That said, as much as Joomla is a bitch, I wouldn't change it. It really does wonders for this site.

Then there is the workflow. It's not great. Coding at home or work, I fire up VS, point it to the webserver I want, edit file, save, done. With radmon I have to open my local copy, edit, save, upload to radmon via ssh, backup old file, copy across as dev file, test, if ok copy over to live file. All of the ssh stuff is all command line, and I don't know nix very well, at all. For small changes and things it's fine, but for big changes it's a bit of a PITA. I generally work by making small changes, then test. Then more small changes, then test, and so on. If I am building a function I'll be testing it many times at each procedural part, then commit the changes to live. This requires a lot of file transfers and a lot of moving files with ssh. It can make a relatively small task take a lot of time.

Finally (well done if you didn't TLDR) there is the charting software. On the page I showed above (my radon monitor) it uses highcharts to draw the charts. It's not free, and not cheap either. I'm running an old version that was free for personal use at the time. I don't know if we could use that here, or could apply for a free license with highcharts, but it has to be renewed every year with them. What if we implemented it and in three years they said, no? There are probably alternatives, but that would mean learning something new and that takes time. Or buy a lifetime license? But they aren't cheap. Last time I looked I think it was about $800 for one site.

So those are the things I want to do, and the reasons I haven't done already. I hope that answers your question, and adds a bit more.

Oh, yeah, I nearly forgot. I want to look at sorting the API to better protect radmon.org, and let the radmon users get their features back. I'll most likely be sorting that first.

Not easily, other than what is available on the site right now. Just looking at the data, I'm not sure it is actually processed. It may just be chopped up. If I cross reference two submission times form two different CSVs I see the same CPM.

Maybe download them all and splice them together? If you let me know what you want to do with the data I maybe able to help some more.

I have just found this:  https://github.com/leeoniya/uPlot

This most likely could be used instead of highcharts and to replace radmon's graph generation system. The charts would only be rendered when someone looks at them, saving loads of CPU cycles and memory, and storage space for the current CSV's and graphs. It's open source and radmon will fit well within the MIT license agreement. This is what steadramon is using for his ESPGeiger network station pages:  https://stations.espgeiger.com/  which is where I found it!