Denied Investigate (& Implement) A Method To Externally Read VTime Data (Or Otherwise Get Player Hours)

[Emilia Foddg]

What does this suggestion change/add/remove:​

Spoiler: What the suggestion seeks to change & add.

If at all possible, implement a system that allows things like Google Sheets to retrieve the hours that SteamIDs have been on a job for, in a given timeframe (For that day would be a good minimum, but given the way activity checking works (at least in (UK) E-11), most preferred would be to specify hours, but make that part of the call optional).

The process would ideally look something like:

1. A department's roster makes an API call to the server (I think we would need some kind of basic support on how to set up this call, maybe a guide if possible?), provides a timeframe, a list of SteamIDs and I guess some kind of ID that refers to departmental/regimental jobgroup (E-11, Nu7, DEA, etc.)?, i.e.:
   For the date of 01/01/1970 [and optionally, between the hours of say, 14:00 & 20:00], provide the hours on E-11 for STEAM_A:X:XXXXXXX, STEAM_B:X:XXXXXXX & STEAM_C:X:XXXXXXX.
   
   
2. Server returns the date, SteamID & the hours that SteamID has played that job during the supplied timeframe (without including time marked as AFK if possible), i.e.:
   01/01/1970, STEAM_A:X:XXXXXXX, 3.85
   01/01/1970, STEAM_B:X:XXXXXXX, 0 [maybe don't return ones with 0 or less hours (yes, "less than 0" sounds stupid, but is probably entirely possible, knowing all the systems we work with)]
   01/01/1970, STEAM_C:X:XXXXXXX, 0.75

Alternatively to reduce API calls, there could be a spreadsheet resource made available to departmental/regimental roster managers, in which the server regularly posts that info, so every few hours it would spit out date, jobgroup ID, SteamID and hours - Which could then be read from by departmental/regimental spreadsheets:

01/01/1970, E-11, STEAM_A:X:XXXXXXX, 3.85​

01/01/1970, Nu7, STEAM_Z:X:XXXXXXX, 11​

01/01/1970, DEA, STEAM_H:X:XXXXXXX, 6.7​

01/01/1970, E-11, STEAM_B:X:XXXXXXX, 2.6​

and so on.​

​

Departmental and regimental spreadsheets would then from that, just look at the ones relevant to their department/regiment. This method would mean not having to distribute the method for making API calls to the server and decrease the likelihood that it could be abused by a malicious actor.

Has something similar been suggested before? If so, why is your suggestion different?:​

Spoiler: Relevant past suggestions to this suggestion #1

Closest previous suggestion I could find was this accepted suggestion from Kwan (presently in the Dev Tracker Backlog) about making /vtime display job-specific time - This suggestion is sort of querying and if possible, suggesting to create a way of externally retrieving a given SteamID's time spent on a job (likely via an API call). Additionally, their suggestion reasoning is more about presenting that information in applications for positions, whereas mine is about using that data in activity logging.

Possible Positives of the suggestion (At least 2):​

Spoiler: Benefits of the suggestion (7 points listed)

The option of being able to automate activity logging in this manner would improve the following for departments that choose to implement it:

• Increased appealability of being in a department/regiment & Convenience for the individual - Where some departments have to currently manually log hours with forms (or however other departments the track activity do it; I know (UK) CI right now uses the built-in regimental activity thing that shows how many days ago someone was online which really only works for them, as there are only CI jobs in the CI faction (assuming that only tracks when you were last in the faction, if it checks when you were last on the server, then... I mean it still works for them to see if that person is active, but it's still imprecise to the level where it's inappropriate for some use cases, such as in (UK) E-11 where we want to make sure you're active on the job)), this expectation placed on players from department leadership to regularly clock in hours typically discourages them from wanting to stay in that department/regiment, as the chore is too bothersome for them, which prevents that department from having as populous as it could be and is ultimately an impact on server health - Players may also have commitments to other roles and positions they have in the server, especially staff (Draw a Venn Diagram of staff members and MTF operatives and be surprised how populous the middle section is) who have to do a lot already without the added responsibilities of being in an MTF (an in UK E-11, logging your hours is one of those responsibilities). Implementing automation of activity logging overcomes this issue.
  
  
• Improved means to track activity on a job - In addition to current departments that track activity, the possibility for automatic tracking enables potential solutions for other departments that would like to track activity, but are already doing too much and would find the prospect of having to manually log hours otherwise unwieldy or unreasonable to implement.
  
  
• Harder to fake hours - Switching from manual hour logging to automatic hour logging means that someone can't forge the amount of hours they played for, the later point about abuse not withstanding.
  
  
• Impossible to forget to log - Can't forget to log hours if they're done for you.
  
  
• Decreased/complete elimination of roster griefing - Since it would be based on an in-game activity and you can't manually log hours after the switch, someone (especially any discharged person that still has the form link - or anyone that's like otherwise received the form from being given the link - ideally people shouldn't just be distributing activity form links like that for this reason but it's not like we can stop someone from just flagging d-class and throwing it out in chat for people to grief: i mean, if we (the CO team) find out that someone's doing that, that's 100% a blacklist, but again, we can't stop it from happening) can't then just fill out the activity forms with IDs of people currently on the roster and a bajillion hours. This has been an occurring issue with (UK) E-11 and (UK) Nu-7 in the past and so far our only solve is to delete the relevant logs; Recently we (E-11) ended up just making a new form altogether because it became an increasingly recurring issue. And of course, new official document, new ownership transfer request to Ventz. Which leads into my next point:
  
  
• Less document transfer requests for Ventz - So ignoring the one spreadsheet for the alternative method of getting that data to the departmental spreadsheet as detailed above for now (which Ventz would also need to own), Ventz has to own every official form or GDrive resource, including things created for things like activity logging. Again, not sure how other departments do it, but it's likely that Ventz needs to own that material, too.
  
  
• Less VTime check sits for staff - Frees up staff time to do other important things and might also make staff work less dull if they're getting a bunch of these.

Possible Negatives of the suggestion:​

Spoiler: Drawbacks of the suggestion (5 points listed)

• Dev time to implement - Might not even be possible, fully understand if so.
  
  
• Ability of department to implement this into their roster if they want to - Ideally this shouldn't be the case with a well-written enough guide, but I get that not everyone can roster good.
  
  
• API calls - Depending on how it's implemented on the roster side, might be too much traffic. Would have to be set up right and responsibly. Also depending on distribution, that API call could be used improperly and is an avenue for a DDoS attack.
  
  
• In-game abuse - It would be easier to cheat the system by just sitting AFK/otherwise doing nothing on a job for the required amount of hours instead of doing what they're supposed to; But the onus for catching and dealing with this is on the individual departments - Same as when if someone tries to fake their logs. This is an IC issue and should be dealt with as such.
  
  
• Underutilisation - It's possible that not enough departments would use this system to warrant its implementation.

Based on the Positives & Negatives, why should this suggestion be accepted:​

It's time to discuss the elephant in the room.

At least in UK, the biggest thing holding a significant portion of people back from joining departments/regiments that have activity requirements, is after putting the hours in, having to log those hours with their department. We recently cut down our activity logging form in UK E-11 to just date, SteamID and hours a few months back and that has improved things, but the prospect still seems unsavoury to a lot of people. I have had recruits actually ask me if I was joking, when I told them that they have to log hours.

If this is at all possible, please pursue this with urgency. I get this sounds weird coming from the best performing department on the UK server right now and while I'm always looking for ways to improve, I'm also looking at things holding us back. This is one of them.

 

[Alpa]

Rather than having an endpoint for vtime specifically, an easier to implement solution would just have an endpoint that returns a list of the players on the server and what jobs they are on, would require more coding on the players end but it's still not really that hard. Other endpoints could return lists of people in a regiment and their ranks and things of the sort for automated rosters, though for the purpose of monitoring playtime the easiest solution imo is just returning a list of what jobs people are on on the server.

So the example endpoint would return:

{
  "players": [
    [
      "xxxx", // [0] rpname (optional)
      "xxxx", //    [1] steamid
      "xxxx" //   [2] job
    ],
  ]
}

then the regimental one would be server and regimental specific and return the following.

{
  "players": [
    [
      "xxxx", // [0] rpname (optional)
      "xxxx", //    [1] steamid
      "xxxx" //   [2] rank
    ],
  ]
}

Imo, its a lot more feasible to implement something like this as it cuts down on server dev time and increases the player dev time, which isn't really an issue and would increase the chance of an api being implemented.

Also to prevent misuse a command could be used ingame to generate an API key (need total level 50 or whatever to generate), so that the usage is tied to an ingame steam account and misuse can be dealt with accordingly (or just rate limit it).

anyway +support

 

[Emilia Foddg]

Rather than having an endpoint for vtime specifically, an easier to implement solution would just have an endpoint that returns a list of the players on the server and what jobs they are on, would require more coding on the players end but it's still not really that hard. Other endpoints could return lists of people in a regiment and their ranks and things of the sort for automated rosters, though for the purpose of monitoring playtime the easiest solution imo is just returning a list of what jobs people are on on the server.

I'm concerned about your wording. I specifically wrote the suggestion the way I did, so as to capture any activity that happened within a given timeframe. If the endpoint returns only whoever is on at the time when it's received the request, wouldn't that mean that say:

Someone flags onto say, Nu-7, and does stuff on that job for some length of time. Then after that, they disconnect from the server entirely because they're done for the day or worse, they crash. Whatever the reason.

Then let's say following that, the server receives a request to get-

So the example endpoint would return:

{
  "players": [
    [
      "xxxx", // [0] rpname (optional)
      "xxxx", //    [1] steamid
      "xxxx" //   [2] job
    ],
  ]
}

then the regimental one would be server and regimental specific and return the following.

{
  "players": [
    [
      "xxxx", // [0] rpname (optional)
      "xxxx", //    [1] steamid
      "xxxx" //   [2] rank
    ],
  ]
}

Would the list of what was returned here then not include that Nu7 in the example, since they weren't on the server at that time, even though they were just on?

Furthermore, the reason I said VTime, is because regiments like E-11 and Nu-7 track the amount of hours people are on for as a measure of ensuring activity within the regiment, with (UK) E-11 requiring a minimum of 3 hours over the past 3 days in order to avoid being penalised (and eventually, removed if it continues) for inactivity - And VTime tracks, stores and gives us that information in-game.

So without being able to retrieve a measure of hours any given E-11 was on for, this would not be suitable for use for E-11. I'm sorry if this sounds nitpicky, but I have to stress the utmost importance of specifically being able to retrieve an amount of hours that a given SteamID was on a job for, regardless of whether or not they are presently on the server, as a result of the implementation.

If that is something we can reasonably get out of this and your example just didn't cover that for example's sake, then I'm just being crotchety for no reason - Your post is otherwise wonderful on the matter.

 

[Billy Boy]

Rather than having an endpoint for vtime specifically, an easier to implement solution would just have an endpoint that returns a list of the players on the server and what jobs they are on, would require more coding on the players end but it's still not really that hard. Other endpoints could return lists of people in a regiment and their ranks and things of the sort for automated rosters, though for the purpose of monitoring playtime the easiest solution imo is just returning a list of what jobs people are on on the server.

This would then result in users having to create a script that constantly queries the API, as they will need to ping it every X amount of minutes to check if the user is online to update their playtime.

Let's say we have 8 divisions (cant remember exactly how many there are), each roster would need a script and lets say they decide that every 5 minutes they will ask the API for the currently logged-in users. Over the day there would be 288 API requests per division. In total giving us 2304 requests for the day.

Whilst it's not too bad, it could be made easier on the user as well as removing the need for all of these requests. If the server were to keep track of the current playtime (as it already does; alteration would only be required to make this job-specific) it could just pass this information alongside the initial API Request, in turn resulting in only 1 API request needing to be made per division as they could get interact with the endpoint similarly to this:


This would allow for divisions to fetch specific information on their division members & the division as a whole in one request, this can allow for fancy formatting of a roster and allow for the divisional roster managers to make it dynamically update should they want to put the effort in to do so. (as well as adding a little status symbol for if they are online/offline & showing the avatar of the user alongside their roster entry).

+support for this suggestion.

--- Edit ---
The API could also include information such as: licenses that they are granted, levels (broken down by d-class, combat, etc), and other information that could prove useful to automate away. Nobody likes paperwork - no need to have it when you're trying to play a game.

 

[Alpa]

So without being able to retrieve a measure of hours any given E-11 was on for, this would not be suitable for use for E-11. I'm sorry if this sounds nitpicky, but I have to stress the utmost importance of specifically being able to retrieve an amount of hours that a given SteamID was on a job for, regardless of whether or not they are presently on the server, as a result of the implementation.

Its like what Billy Boy said above, it would require a script to make an api call every few minutes to check who is on and then store all the data on the user's end, I phrased it the way I did in order to provide an alternative to the server collecting job specific vtime. While there is already a suggestion that has been accepted to do this, it has been on the backlog as low priority for 3 months, having it so users have to do all the tracking themselves would allow for this to be implemented easier for the server, however would be harder for the end users, though still not that difficult (It might be difficult doing it directly into a google sheet, im not sure but just running a python script or something it could be really easy).

This would then result in users having to create a script that constantly queries the API, as they will need to ping it every X amount of minutes to check if the user is online to update their playtime.

Let's say we have 8 divisions (cant remember exactly how many there are), each roster would need a script and lets say they decide that every 5 minutes they will ask the API for the currently logged-in users. Over the day there would be 288 API requests per division. In total giving us 2304 requests for the day.

I don't know too much about API programming when it comes to hosting the API, however that isn't that many API requests, even if it was per minute on both servers it would be 12/minute (4 mtf, ci, goc per server), which isn't that much still. Having it track per-job vtime would result in more database reads and writes as well as compute power at the cost of reducing API calls, all the calculations would be done on the server as opposed to the users end. I can't imagine that is easier for the server to handle than the API calls.

Ideally the server could track it, it would make it much easier for the end users, however having the user's track it would be easier for the server developers to implement and allow more freedom for the users to track other job related metrics that may not be specifically tracked by the per-job vtime implementation. My suggestion was just an alternative incase the developers decide it is not feasible to implement the original suggestion due to specific constraints.

 

[Starling6]

So many big words and smarter people then myself.

But adding a API to grab info and build a roster would be amazing from both a Player & Regimental Leader view. Being able to know when you are finished with your requirements and being able to get off when you want too rather then feeling pressured.

And from a leader standpoint; having it automated means we can require actual time on the job and keep tabs on less active people, rather then just using the regimental timer. Alongside not needing to grab 50+ Steam ID's.

From a SL Standpoint; its nice to not have to take 20 sits for VTime checks every other day.

+Support

 

[Emilia Foddg]

I don't know too much about API programming when it comes to hosting the API, however that isn't that many API requests, even if it was per minute on both servers it would be 12/minute (4 mtf, ci, goc per server), which isn't that much still.

On UK, Department Directors are now required to keep track of their activity. They may want to use this. Other departments that may potentially be interested in using this implementation to at the very least, automate their activity status - and obviously for their CL4+ only (which are the only people on their rosters anyway), would include (all rosters linked IIRC are public):

• IA
  
  
• DEA
  
  
• Research
  
  
• GenSec
  
  
• Medical
  
  
• ECAs
  
  
• OSAs (yes, they have separate rosters)
  
  
• Site Administration (shaky on this one)
  
  
• Ethics (extremely shaky on this one)

...Suddenly, the numbers start adding up.

From a SL Standpoint; its nice to not have to take 20 sits for VTime checks every other day.

Oh yeah, you guys take VTime check sits. I mean, aren't those free sits for your quota? Eh, whatever. I'll put it on the positives list.

 

[Billy Boy]

Having it track per-job vtime would result in more database reads and writes as well as compute power at the cost of reducing API calls

The server could efficiently interact with the database, on player leave / job change / on a certain interval update their playtime within the database. All this would do is take the burden of manually reporting it off the user and allow the rosters to just ask the server instead.

It might be difficult doing it directly into a google sheet, im not sure but just running a python script or something it could be really easy

Yeah it would be easy to create a python script for it (with ChatGPT or if you know how to program) however should the API have downtime or should their python program stop working for some reason (internet temporarily goes down) then the playtime would be lost for that division during the time the script is not running.

If the server was to be restarted all information stored in the memory of the python program (in variables, etc) would be lost, so a database would then need to be setup by each division to store this data.

This approach would also have the divisions needing to either purchase their own VPS or by getting resources allocated by CivilNetworks.

allow more freedom for the users to track other job related metrics that may not be specifically tracked by the per-job vtime implementation

This could still be accomplished with the implementation that I was talking about, more ranges could be included if need be. Or you can track the "currently_online" field manually yourself. (still allowing for the freedom)

 

[Corr "Perseus" Vynd]

+Support

Since CG definitely uses some sort of SQL to track your player data like achievements (so that includes vtime), it shouldn't be too difficult to implement a threaded REST API for this stuff. Make a Rust app that would run on a separate thread to query the DB with ratelimit per IP and temp banning of IPs if they query too fast in the situation someone tries to crash it. Alternatively CG could build this into the website to skip this worry. Either work.

Or alternatively alternatively you could have SL generate an OAuth access token that can be revoked for COs, fac leaders, etc etc that need this sort of data, this removes a bunch of the worry with performance.

 

[Bohemia]

+ Support
The world if I could VTime check reg jobs

 

[Mars]

+ Support
The world if I could VTime check reg jobs

+support

 

[Starling6]

Oh yeah, you guys take VTime check sits. I mean, aren't those free sits for your quota? Eh, whatever. I'll put it on the positives list.

. . . by the time we hit SL, Requirements are sort of null

 

[cinco]

No idea what any of this stuff means but cool

 

[Emilia Foddg]

No idea what any of this stuff means but cool

I'll ELI5 it for you:
As things are, some departments or regiments (I'll go with my example of UK E-11 again since I'm a CO), require all members to manually log the hours they play E-11 (or w/e job, you get the idea) for, so that we can purge inactivity.

If this is accepted, implemented into the server, then implemented into said department/regiment, then ideally, no-one in that department/regiment would ever have to do that ever again.

 

[Alpa]

If the server was to be restarted all information stored in the memory of the python program (in variables, etc) would be lost, so a database would then need to be setup by each division to store this data.

This approach would also have the divisions needing to either purchase their own VPS or by getting resources allocated by CivilNetworks.

You can get a pretty powerful VPS for free with oracle always free tier, runs 24/7, you could store data there in a json file or something, you could also just use any free database provider such as mongoDB, firebase ect.

The server could efficiently interact with the database, on player leave / job change / on a certain interval update their playtime within the database. All this would do is take the burden of manually reporting it off the user and allow the rosters to just ask the server instead.

Yeah, like I said I don't know how the current VTime implementation works but in a void it seems like individually tracking every job/department on the server for all users could be difficult, especially with a jr developer replying stating the same below.

Implementing this in a decent way would be challenging though (especially considering the timeframe thing & making it storage efficient).

Like I said, my alternative is worse for users, it is just throwing out a seperate idea if the original suggestion cannot be done for any reason.
Though I imagine the issue with a lot of API calls is sort of null, the api calls would return less than 32KB of data, I guarentee the server could handle under 30 api calls per minute, it isn't that computationally intensive.

 

[Emilia Foddg]

If the server was to be restarted all information stored in the memory of the python program (in variables, etc) would be lost, so a database would then need to be setup by each division to store this data.

This approach would also have the divisions needing to either purchase their own VPS or by getting resources allocated by CivilNetworks.

You can get a pretty powerful VPS for free with oracle always free tier, runs 24/7, you could store data there in a json file or something, you could also just use any free database provider such as mongoDB, firebase ect.

I mean, again, can't speak for other departments, but in E-11, we currently collect activity logs in a spreadsheet table, linked to a Google Form that just spits its results out into it. And we just retain that data.

Like, we have the storage method, I want a method of getting the requisite data into the storage method.

 

[Cary “Raven”]

+ support

This would be a life saver. Very cool

 

[Snake]

+support
On top of making logging hours potentially easier it also makes it easier to check if someones logged the right hours or made a mistake etc

 

['Scorpion' | 'Hitman']

+support I HATE EVERYTHING TO DO WITH LOGGING

 

[Emilia Foddg]

+support I HATE EVERYTHING TO DO WITH LOGGING

holy shit famous A-1 major hitman

 

[emilsnat]

+support

quality of life changes like this are always worth the dev time it takes to implement them
It'd be nice to have this accessible on the portal rather than make it an API