Did You All Get An Update From Steam Yesterday?

Torf
Thanks for the very informative post.
I guess the only difference between your install and mine is that I do NOT have any modded files such as on your system.
As the vast majority of TSC simmers probably do not have these modded files they did not experience the issues that you and the OP described.
As we do not have access to the TSC core programming code we have little idea of what modifying TSC files might do in its relationship to the Steam checking processes.
Has anyone raised a complaint with STEAM?

 

Peter, the TSC code has nothing to do with it - it's just Steam verifying what should be there and what not via the depot index and file checksums and manifest IDs.

Yes, I have many mods. Almost every cab camera is adjusted, engine bins adjusted for better controls or some US locos which had 9 notches fixed, all the coupler fixes, and a lot of other bugfixes.

Of course on a vanilla installation you'll hardly notice Steam verifying. Due to the .ap file distribution the check is very quick, only one checksum instead of ten thousands.

 

Now that you mention it - when the .ap system was introduced circa 2013 some simmers (not all) experienced (modified) files being deleted outside the .ap file structure.
There was a lot of discussion at the time on UKTS, RWA, the old STEAM forum, etc but opinion was divided as many of us were not affected.
We knew little of the .ap file structure back, and I carried out quite a few Sysinternals apps analyses to eliminate windows issues. There were none!
The discussions petered out after a time without any real resolution, and now it is still happening!
I wrote this in 2013:

and

 

Yes, that's how it works.

The bad thing is now Steam does it by itself, without asking, when installing new content - not always though.

Again, I praise the way ETS2 is structured. You never touch the game folder itself. You have your ETS2\mod folder in your Documents.

The principle is the same there, recreate original folder structure and put modded files in there, zipped or unzipped. But this folder is not affected by the verification process because it is separate from the game files.

 

So the upshot is that forced verification is something that DTG can’t prevent, because it’s done automatically by Steam, but they can obviate by use of the two folder structure as used by ETS2?

Is this a solution that TS can readily be reworked to, without causing (multiple) collateral issues that need fixing?

If it’s a non-trivial matter to do, there is also always the unfortunate balancing act of staff and money resources v size of users it would affect….

 

I don't think that's an issue for DTG, because it is apparently a problem for a few users only. Most people have not modified their files, and the verification doesn't touch added 3rd party content. (Except for addons that manipulate or replace original files - AP Weather EP TimeOfDay files, Signal textures and its modded Kuju Core files will get removed for example.)

Just an example, I've packed the NZ Midland Line and Taurus assets into ap archives which significantly speeds up loading time - Steam wants to revert it back of course. (I've created a thread on that some time ago, just a reminder you have to check for a few things before doing so, some loco and signal scripts refuse to work when inside ap's - see Dresden assets that come as an ap plus an unpacked folder with the signalling stuff)

I've found my workaround - using a separate minimal clean TS core install that I'm using when purchasing Steam content, and then manually copying the new files to my full 500+ GB install. This involves manual editing of the appmanifest_24010.acf file though, adding the new DLC manifest to the list.

Edit:

I'm still trying to figure out if there's something on my end which triggers the process. The "verify_cache_key" was an experiment, but it's not related to the issues here.

I will check if maybe there's a mismatch between the DLC manifest IDs in my appmanifest and Steam's database - I remember some time ago there was an update to a route (edit: Montana Hi-Line) that did not download automatically for me, I had discovered that in the Announcements subforum, and applied the update via un/reticking the DLC.

 

So I found something odd today: it seems some add-ons trigger a Steam validation, some add-ons don't. I bought two add-ons today: Corris Railway and B&O Mountain Subdivision. I bought them separately, and Corris Railway installed without triggering validation. B&O Mountain Subdivision triggered a validation, so I paused the process before it damages my heavily modded installation. I renamed my whole Railworks folder, and installed TS from scratch. I unticked both Corris and B&O before installing TS, and Corris installed without any issues again. B&O triggered a verification again with a clean TS install. It must be something to do with certain addons, but can't figure out the exact reason. B&O is more than 5GB, maybe it's something to do with the package size?

I also have to add that I launched the freshly installed TS once before attempting to install Corris and B&O.

 

I reinstalled Mountain sub a few days ago (to get rid of a mod). Unticked it in the steam client and then deleted the remaining files. Then ticked it again in steam to reinstall. That didn't trigger an verification.

 

Investigation finding: B&O modified the following route: 00000006-0000-0000-0000-000000000000 (whatever this route is). This must be an old Kuju route, not B&O itself. The file creation dates have all changed (I only know this because I used an app called BulkFileChanger, which resets the creation time of all files). So it was easy to spot the difference. This was during the second fresh install, as I feared resetting file creation times might trigger a verification. Corris, again, installed without any issues.

 

00000006-0000-0000-0000-000000000000 is the original Castle Rock Railroad which came with RailWorks. It was withdrawn in 2014 or 2015 iirc and rereleased on the Workshop (with new GUID 85e59955-ee4d-41f4-bfbd-b84444e53a1a).

Even if you do not have the route, some loco DLCs put scenarios in there. (From memory I think one of the GE 44-tonners for example) and if you've changed or deleted them, the Steam verification brings them back.

 

I must own the original then, bought the game back in 2009 if I remember correctly. Could this be the reason? I can't see how these two things are related, though.

 

kurtosizm
The Castle Rock Route 00000006-0000-0000-0000-000000000000 should have converted to an .ap file in May 2017 along with a scenario pack. So not sure that it was withdrawn in 2014/15.

 

Peter, where did you get the information?

Castle Rock was withdrawn from the store - so it's not available anymore for customers since 2014 (owners can still download it of course)

The depots containing the files are the following (split up in route and scenarios, as it is for the classic CP, GWML, ECML, SDJR and HagenSiegen). The route files come unpacked. As I see in your installation, you have BOTH a packed MainContent.ap AND unpacked folders, which is a bit odd.

Steam hosts the route as a bundle containing Hedborough North, Castle Rock Railroad and Seebergbahn. There's no MainContent.ap.

Also the DD40X does not come with these scenarios anymore sadly. Likewise the NS SD40 HN scenarios for CR have been removed also - even for owners! I was thinking to remake them for the WS version of the route, but I'm not sure about that copyright thingy...
So make sure to make a backup cause you'll not be able to get them back in case of reinstalling.

Railworks 3 LegacyRoutesFictional
https://steamdb.info/depot/208305/

Railworks 3 LegacyScenariosFictional
https://steamdb.info/depot/208315/

In addition, there's a lot going on at the moment. DTG are stripping the names from their depots in a bulk operation, so they'll all bear the "ProductNumber" name... (does NOT affect the DLC names though - only their associated depots)

Example:

 

Torf
I presumed that it was updated automatically from the original version.
Other than that I just do not know!
Does anyone else have the same update?
This is the .ap file opened





Do you think it is from DTG? Sorry it misplaced the images - hope you can decipher

 

I can only look up SteamDB. It does not list the MainContent.ap - browsing through the history may reveal a change that had happened though. Unlikely they'd change from .ap to unpacked - even considering that my dicovery with the wrong CastleRock Assets packaging has been ignored by DTG, although I had pinged DTG Jamie

 

This is the Castle Rock on mine. I have just installed it as I didn't have any of the legacy DLC installed.

 

I have had TS since TS2012/Railworks 3.
I used to have Castle Rock installed but at some point it was removed by me - maybe when changing drives.
But it is not in my Downloads as Castle Rock

 

Peter, as I wrote before you'll find it (together with Hedborough and Seebergbahn) under the following names in your DLC manager:

Railworks 3 LegacyRoutesFictional
https://steamdb.info/depot/208305/

Railworks 3 LegacyScenariosFictional
https://steamdb.info/depot/208315/

 

I looked at my DLC and everything under Legacy is ticked - with an installed date of Dec 24 2011

I unticked and re-ticked and the route has downloaded

Interesting even though it comes with a MainContent.ap everything in the file has been extracted

 

I guess because it wasn’t legacy when you originally ticked in in 2011, but recognised as that when ticked now?

 

I don't think the ability to organise the DLC like this was in the Steam Client then - I might be wrong

 

Everytime you go digging you'll find something

Indeed the files are doubled - They packed the .ap and forgot to delete the files outside... and the CastlerockAssets.ap is also out of order

Peter Hayes trevkiwi Now I see where the .ap came from

 

Torf
There is something strange with the CastleRock .ap version. The ASSETS .ap file has its own entry ie it is NOT in the DTG or RSC folder, which is unusual.

 

Peter Hayes

I've already pointed that out, got mostly ignored it seems. The path is unusual but correct (Routes refer to Provider "Castlerock" - Product "scenery" ) but the .ap is incorrectly packed.

For years I did not discover it until my old eyes spotted milk bottles (the whistle signs) along the route, then I checked the files - and fixed the issue. That's why I'm avoiding verifications - because it brings back all the bugged stuff. (Checking, I have already a collection of 2000 files I've fixed)

Considering the above discussion, it seems all of this was done by somebody at DTG not really knowing what they were doing... but then it's not a big deal making mistakes - not correcting them when you're aware of is.

https://forums.dovetailgames.com/threads/castlerock-assets-must-be-unpacked.64180/

 

My CastleRock has been an .AP file for a few years now, several independent 3rd party route developers utilise larkspurstation on their routes, so, it's always been a 'must have it installed' asset file for me, although, I openly admit to not actually driving on the Castle Rock route itself for a long, long time.
And I would assume that all our installations have been on different Drives for a long time. I had my game on an SSD once, it was N;Drive\SteamLibrary\Steamapps\Common\Railworks, my actual Steam installation was first put onto my C;Drive in 2009, when I obtained CallofDuty, etc, but, I now have all my computer games spread across 3 Drives, so, every letter is different and the path wording is different, Steam, or, SteamLibrary, etc, etc.

I have now trimmed my hard drive down slightly where TSClassic is situated, as I have other railway simulators installed on the same drive, and made every .AP file have its own folder and every AP file has then been extracted and those files sit in another folder, obviously with a slightly different name change, so, if this continues to happen, then I can pause Steam and go into my Content/Assets folders structure and let the verification choose the unopened .AP files which would allow the verification to skip over all the .AP files hopefully as they are all still intact, this will take about 15 minutes and then allow a 10 minute verfication instead of a 3 hour one, which is intolerable.

Doesn't anyone extract their RouteProperties.xml file from all installed routes using TSTools? I was under the impression that if I wanted to clone a route this file had to be available for the procedure to work properly, and if I wanted to rename a route when opening the game in editor, it would allow the changes to take effect, and you can see the changes if you opened the xml file in the Route folder afterwards.

Cheerz. Steve.

 

Thanks for your reply, I'm not tech savvy so can only rely on simple techniques to get me through playing PC games, I have a reasonably good memory for this game folder's structuring and files over the last 18 years nearly, since Kuju days anyway, so, I opted for an easier approach to this problem if it happens again, it was time consuming to complete but is now all ready if this happens again.
I haven't contacted Steam as I don't know what to tell them really, I understand that the verification of files exist on every game that we obtain on the Steam platform, unfortunately your game seems to be a major sticking point to all the others I own because it is so huge.

Cheerz. Steve.

 

Technical detail. If you get a key for a version of RW/TS that provides the three fictional routes, they CAN be obtained with all scenarios in the package for all routes. That said, they are very close to identical to Workshop versions, posted by Matt personally after the change in direction with the product. The scenarios can be moved to the Workshop version, as well, and Workshop ones to Steam versions unless they use a specific watering point added, or something like that, in the free versions.

 

Something mentioned on UKTS
If anybody suspects that a verify operation has been invoked, check for the presence of this zero byre file--- ..\Steam\SteamApps\common\RailWorks\verify_cache_key

Deleting that file before running the game has been shown to cancel the removal of files

I don't have that file but it might be worth people who had this problem checking it out.
martschuffing , ex_railwayman - worth a check

 

Peter, this has already been discussed in this thread.

https://forums.dovetailgames.com/th...from-steam-yesterday.62394/page-2#post-593254

This file is shipped with the core files (depot 24009) and is a way for the developer to force a verification. Once that has happened, the file is removed.

So it happens:

1. After a fresh install
2. If any verification restores the core files (this depot) due to changes made (AP Weather changes core files for example)



Bit off-topic, but I notice on the Steam ETS2 forums that people are having trouble with corrupt files after updates and need to verify often. I'm not sure if this is due to cheap SSD hardware not dealing well with the delta patching process Steam uses. Never had corrupt files ever on my HDD.

 

My bad - I missed that post

 

Thanks all. However, I have taken my railworks folder out of the verification regime by installing on a seperate drive, everything good now.

 

I'm having a little issue. Just purchased the following DLC:

https://store.steampowered.com/app/65216/Train_Simulator_EdinburghGlasgow_Route_AddOn/

It haven't done a full verification for months, so I thought this would just be a normal installation. I was wrong, and it triggered a full verification, so I paused it as soon as I saw the process, and deleted everything from the steamapps/downloading folder. My original appmanifest_24010 looked like this (the bottom of it):

"DlcDownloads"
{
"24091"
{
"BytesDownloaded" "0"
"BytesToDownload" "141648"
}
"65216"
{
"BytesDownloaded" "5856"
"BytesToDownload" "436152336"
}
}
"UserConfig"
{
"language" "english"
"DisabledDLC" "24080,65231,1230271,258643,448190,376962,376961"
}
"MountedConfig"
{
"language" "english"
"DisabledDLC" "24080,65231,1230271,258643,448190,376962,376961"
}
}

I highlighted the new IDs in red.

It also had the following line at the beginning:

"AppState"
{
"appid" "24010"
"universe" "1"
"LauncherPath" "C:\\Program Files (x86)\\Steam\\steam.exe"
"name" "Train Simulator"
"StateFlags" "1076"
"installdir" "RailWorks"
"LastUpdated" "1683128935"
"SizeOnDisk" "209322920572"
"StagingSize" "679683995"
"buildid" "11146675"
"LastOwner" "76561198019173852"
"UpdateResult" "4"
"BytesToDownload" "0"
"BytesDownloaded" "0"
"BytesToStage" "0"
"BytesStaged" "0"
"TargetBuildID" "11200143"
"AutoUpdateBehavior" "1"
"AllowOtherDownloadsWhileRunning" "0"
"ScheduledAutoUpdate" "0"
"FullValidateAfterNextUpdate" "1"
"InstalledDepots"
{

I changed this to 0, and quit Notepad. I duplicated the file, and modified the bottom if it as follows:

"SharedDepots"
{
"228983" "228980"
"228989" "228980"
"228990" "228980"
"229020" "228980"
}
"UserConfig"
{
"language" "english"
"DisabledDLC" "24080,65231,1230271,258643,448190,376962,376961,24091,65216"
}
"MountedConfig"
{
"language" "english"
"DisabledDLC" "24080,65231,1230271,258643,448190,376962,376961,24091,65216"
}
}

I deleted the DlcDownloads section, and added the newly purchased DLC IDs to both DisabledDLC section.

I restarted Steam, but it still wants to do a full verification. Any way to stop this from happening?

 

Be sure that StateFlags is set to "4" which equals "everything ok". This number basically tells Steam if there's an update pending or ongoing and paused, or other things.

Delete everything in steamapps\downloading\24010 and steamapps\temp\24010

As the BuildID is different from TargetBuildID expect an update to be triggered - based upon the individual DLC ManifestIDs. (The BuildID is incremented with every update made to any of the depots)

Also make sure there's no dummy file named "VerifyCacheKey" in your RailWorks folder - if there is, delete it.

Note: Make sure Steam is not running while making these changes or Steam will revert them instantly (as the appmanifest is cached in memory and will update the local file automatically)

Steam appmanifest state flags bitmask (can be combined like 1024+2=1026 (Update reqd and started))

Code:

AppState
StateInvalid         0
StateUninstalled     1
StateUpdateRequired  2
StateFullyInstalled  4
StateEncrypted       8
StateLocked          16
StateFilesMissing    32
StateAppRunning      64
StateFilesCorrupt    128
StateUpdateRunning   256
StateUpdatePaused    512
StateUpdateStarted   1024
StateUninstalling    2048
StateBackupRunning   4096
StateReconfiguring   65536
StateValidating      131072
StateAddingFiles     262144
StatePreallocating   524288
StateDownloading     1048576
StateStaging         2097152
StateCommitting      4194304
StateUpdateStopping  8388608

 

So I did the following (thanks for the above, it was very useful):

1. Closed Steam.
2. Deleted everything from the steamapps/downloading and steamapps/temp folders.
3. Edited the StateFlags string, it was "1076", changed it to "4" and saved the appmanifest file.

After the first restart, it was "1174". Did everything again, now it's "6".

This is how the beginning of the appmanifest file looks like at the moment:

"AppState"
{
"appid" "24010"
"universe" "1"
"LauncherPath" "C:\\Program Files (x86)\\Steam\\steam.exe"
"name" "Train Simulator"
"StateFlags" "6"
"installdir" "RailWorks"
"LastUpdated" "1683894523"
"SizeOnDisk" "209323633967"
"StagingSize" "679655671"
"buildid" "11200143"
"LastOwner" "76561198019173852"
"UpdateResult" "0"
"BytesToDownload" "0"
"BytesDownloaded" "0"
"BytesToStage" "0"
"BytesStaged" "0"
"TargetBuildID" "0"
"AutoUpdateBehavior" "1"
"AllowOtherDownloadsWhileRunning" "0"
"ScheduledAutoUpdate" "0"
"InstalledDepots"

It looks like it still wants to do a full verification.

 

As long as any of your active DLC's manifestIDs don't match with Steam's database, it will trigger an update (6). Unfortunately this can lead to a verification for reasons not yet known.

There is workarounds for this - you could just rename your RailWorks folder so Steam can't access it. Then check for updates via SteamDB > 24010 > history and the Announcements thread, and apply them on a second installation (with a separate manifest, I keep two versions: one with all DLC active, a second one only 24009 plus the dlc I want to update.) The rest is copying files over and updating the full appmanifest manually.

1174 is 1024+128+16+4+2 - 128 indicates a corrupt file

 

Now something strange happened, as I did the same thing you suggested again.

When I launch Steam, now I don't have a pending update, and the appmanifest looks like this:

"AppState"
{
"appid" "24010"
"universe" "1"
"LauncherPath" "C:\\Program Files (x86)\\Steam\\steam.exe"
"name" "Train Simulator"
"StateFlags" "4"
"installdir" "RailWorks"
"LastUpdated" "1683894523"
"SizeOnDisk" "209322920572"
"StagingSize" "0"
"buildid" "11200143"
"LastOwner" "76561198019173852"
"UpdateResult" "0"
"BytesToDownload" "0"
"BytesDownloaded" "0"
"BytesToStage" "0"
"BytesStaged" "0"
"TargetBuildID" "11200143"
"AutoUpdateBehavior" "1"
"AllowOtherDownloadsWhileRunning" "0"
"ScheduledAutoUpdate" "0"
"InstalledDepots"
{

Is this considered to be the default state? The buildid and the TargetBuildID matches. Is this the right time to tick the new Glasgow - Edinburgh DLC (and hopefully it will install)?

 

That's how it should look like. Make a backup of the appmanifest, and activate the DLC. If successful, backup the new updated valid appmanifest.

 

So I tried, the Turbostar included with the package (separate ID) downloaded without any issues. The route is still triggering a verification. Maybe some IDs are not set up correctly by Steam or the dev, that's why Steam wants to do a full verification?

I paused after the validation (I'm brave), and it started downloading all sorts of files to the downloading folder (e.g. Wherry Lines related files) that are not related to Edinburgh - Glasgow. So it seems my game files are not changed at this stage. Possibly once everything is downloaded into the downloading folder, that's when it starts updating and restoring modified content. On the other hand, this is how my appmanifest looks like at the moment.

"AppState"
{
"appid" "24010"
"universe" "1"
"LauncherPath" "C:\\Program Files (x86)\\Steam\\steam.exe"
"name" "Train Simulator"
"StateFlags" "1206"
"installdir" "RailWorks"
"LastUpdated" "1683894523"
"SizeOnDisk" "209323633967"
"StagingSize" "970038602"
"buildid" "11200143"
"LastOwner" "76561198019173852"
"UpdateResult" "4"
"BytesToDownload" "457655024"
"BytesDownloaded" "104784"
"BytesToStage" "1299277835"
"BytesStaged" "9954053"
"TargetBuildID" "11200143"
"AutoUpdateBehavior" "1"
"AllowOtherDownloadsWhileRunning" "0"
"ScheduledAutoUpdate" "0"
"InstalledDepots"

I guess I will rename the Railworks folder, have a clean install with only the Edinburgh - Glasgow route, and then I'll drop these files in the backup. Should work I think.

 

Sorry, Peter, I haven't been on here for a while. yes, I do have that 0kb file, but, I left it alone as I have no idea what it is for and knowing my luck it might be important if I do delete it, which has happened in the past, so, I haven't touched it.

Cheerz. Steve.