Protagonist, Can You Clarify What We May Expect From Bug Fixing?

At the technical forum a discussion is running on the HH "Cumberland Charge" scenario. It is no longer working properly and cannot be completed. A bug was reported for this and the answer was, that it will be fixed as soon as the cause is known.

To me, this feels like a useless answer. I would like to know what we may expect if a new DLC, let's say a route is issued.

What quality can we reasonable expect fro a DLC when it is launched? How does testing work and what is the QA department doing? Are there checklists? Are all issues solved, or may there be known issues and can these be shared with the customers?

What policies are used afterwards regarding feedback? Is there a minimum quality level to attain or is it just budget driven? Does it depend on the number of complaints?

For me it is a serious question. If five scenarios are promised, I expect five working scenarios.
If there should be 100 jobs, I expect to find 100 jobs and they should be working (at RSN this is still not the case).

I really would like a better view what DTG thinks that is reasonable. It saves time to report issues that never will be solved. It also may remove a lot of irritation because we may somehow agree upon an acceptable average quality standard. I am aware quality has its price, but I have never been asked if I want to pay for better or maybe less quality.

 

I have to give you a non-answer on this one because I can't give you anything concrete.

QA test to a plan to ensure that everything we release is in a fit state for players. The majority of problems that would be considered a bug are caught and dealt with appropriately. Obviously this isn't always the case and occasionally and issue will escape into the wild. In addition, is highly likely that 1,000 players handling a route will uncover nuances not picked up by QA, that's a straightforward time factor.

So far, everything above is reasonable, and normal across all game and software development.

Where you are absolutely correct is that if 5 scenarios are released as part of a route you should expect all 5 to be working as designed, and in some cases this has not happened.

I'll raise this internally and see if there is something concrete that can be put in place. Ultimately, wait and see if there's any tangible action rather than waiting for a strong sounding statement, but I'll endeavour to get the latter anyway.

 

I would never expect that everything is tested, this should always be in a reasonable manner.

As an example those 5 scenarios per route, i expect to be in full working order, because those 5 scenarios can be tested in a very reasonable time frame.

Services are another thing, i would never expect your testing to drive those sometimes over 100 services.
But since they are mostly a copy of just 5-6 templates, i expect that at least one of each gets tested.
That way things like the freight-routing starting in Wuppertal would have been caught, because that's completely unreasonably bonkers.

If there are problems in a few services that have not been caught, that would be no problem, IF they would be fixed within reasonable time. (<-There lies the problem...)

New releases are never bugfree, only fools expect that. But what i expect is that found bugs are fixed within a reasonable window. Which would be within 2-3 weeks, not months/years later when the interest in that route has diminished.

 

I don't own the route, but I'm tired of seeing people complaining about game breaking bugs in the NEC route in this forum.
Since I joined this forum I've seen people reporting that and it had probably been reported for a long time prior to my join date.

One would see this and reasonably assume that's how they're going to treat their customers in the future.

 

What I don't understand is why we have all these bugs like the LIRR tunnel glitch in tsw for months and nobody has bothered to fix it. It shows that you guys don't test your routes hard enough. It's ridiculous how you say put it in the fourms but nobody fixes them like rro crashing when you approach Hagen Wehringhausen. If it was a day one bug I could understand, but bugs that happen for months with no fix in unacceptable. Please get them fixed and please actually test your DLCs instead of releasing them just to make intial profit like Canadian National cause that was an absolute joke. I was so close to buying it too but I'm glad I didn't

 

I think the main criticism is that some of the bugs that get through are very very obvious, like the Class 31 Cement Circle issue after the Class 20 release. How was this missed? I know time is a factor, but surely every existing scenario on a route should be retested when something new is added to that route. Had the scenarios been played in this way, the cement circle issue would have been identified immediately given the large number of people affected across all platforms.

 

I'll assume that they don't have a proper debugger (that's essential) or in this case the human debugger might have fallen asleep during practical examination. ^^ Or they also may have not done their inspection properly ... or didn't bother doing that in the first place. How about considering it for the future?

 

I have no idea if this has ever been discussed in the past, but I'm sure there would be several dedicated TSW owners who would be prepared to beta test a new route, e.g. "Please Test the 8:40 am service to woop-woop. Drive it two or three times, at different speeds, and with whichever Loco(s) can be used. Muck around, see if any weirdness happens..." Perhaps these beta testers would be known users, those who tend to buy practically every DLC anyway, and are prepared to sign a NDA. It wouldn't take long to make sure that every schedule and scenario in a DLC works if this were done.

On the other hand, if there's one or two staff in DTG that are assigned this task, I can see how "we didn't have time to test everything" becomes an issue. Plenty of developers use the power of the community to get good returns this way. Of course once it's released to several thousand users, other bugs might pop up. But the obvious ones should have been squashed.

 

I think that's a terrible trend. Companies need to pay testers in order to deliver a polished product to their users.
We should not pay money to test products we're still going to pay for.

 

Fair enough,

Typically, the beta testers would get the DLC free - I know, that doesn't seem much reward - but they tend to be die-hard users of whatever the sim/game is. The opportunity to provide feedback, and end up with something that runs well for everyone, is reward enough for them. There are some wargame devs out there that have a huge list of beta testers (a count of over fifty in one I recall). Having said that, those tend to be very small operations of about five staff and they aren't exactly rolling in cash. (Don't know anything about DTG size or sales totals).

I'm not suggesting this is "the way forward" at all. Given this thread I just wondered whether "community beta testers" has been discussed.

 

If possible, that would be a great idea.

 

Something NOT like this (after the MSB daily update) for example.

 

Think these three posts cover the position on bug fixes.

As for testing, definitely need more people testing who are not involved with the development, they are too involved with it and it can be difficult to step back and actually see what's happening. Running some debugging routine might tell you that some code is wrong, but it's like a spell checker, sometimes things get missed. There's no better way to test something than to let someone actually play it for a couple days.

So a bigger pool of beta testers would be great, but there's no point just accepting people to the beta program who buy everything and accept everything they say as gospel, you need the people who are critical and who will pull it to pieces, they want it to be bad so will find far more issues.

It doesn't mean that the software will be bad or everything will need or be fixed but I guarantee the "haters" will find more issues than the "fans".

 

I suspect a lot of the bugs present at release are already known about by DTG as they will have been tested and the bugs found but there isn’t the time or priority to fix them all before release, or in a lot of cases after release. That may be they case even with some of the more obvious big bugs like scenarios not working. There’s obviously a very tight schedule. I’m not sure more people finding the bugs would improve the situation, the problem is with the time and resources to fix the bugs that are found. Hopefully this will improve with TSW2.

 

It might be time to create the community bug list - which we have discussed before.

If DTG really can't (or won't) offer any clarity or explicit commitment on whether they will fix TSW bugs (yet continue to sell TSW DLC), or indeed TSW2 bugs...... then the community needs a way to track all of the bugs, and get them flagged so that they will get fixed. I am not necessarily talking about involving DTG's QA here.... That part can be.... um.... skipped...... if it is demonstrably not effective.

Unfortunately (as far as I am aware) nothing much can be done for any bugs that are specific to PC or XBox (unless they seriously interfere with Win 10 or the XBox OS), but any major bugs that impact on PS4/PS5 should be fixed, as a requirement of a Right To Develop a PlayStation Compatible Product and market it on PlayStation Store. It would be an unfortunate road to have to take - but if you want assurance of quality, then Sony do provide for this - and they will make it happen.... if it actually comes to point where they have to be asked to intervene.

Very serious issues, like the DB BR 155 on RSN, or the Class 47 on NTP were quite easy candidates to flag to Sony for fixes (and thankfully these bugs were fixed before having to notify Sony's QA team). The Class 31 Cement Circle scenario on TVL is in a similar category of seriousness - so I fully expect that to be fixed before TSW2 releases.

However, a large group of minor bugs would also be flaggable. I propose that we build two public community lists - one for TSW and one for TSW2. If you want to help with this, then I would be prepared to get the ball rolling with an initial short list in a format that can be used, showing the severity of the bugs, and the time that has elapsed as reported but unresolved.

Obviously, I would prefer to trust DTG to handle this sort of thing and act responsibly and reasonably - but if they are going to be vague about future QA and bug-fixing policies, and throw around words like "Preserved".... then it is, unfortunately, probably time for the community to act.

 

Yes, I completely agree with you there. They have made it clear several times that they know about many of the big bugs, as they often get mentioned in streams and such. Like certain crashes, or the tunnels at New York Penn, etc. A few weeks ago I even got an email from them replying to my bug report about the LIRR M3, stating that they could reproduce all the issues I have reported (except one) and they have been documented and sent to the proper team, etc.

So it's not the case of them simply not even knowing about the issues. They know about them, they are just not high enough priority to get fixed.

 

This is known as beating a dead bridge.

 

I used to cheerfully perform a good deal of beta testing for a creative team that built wonderful routes for TS. In return, I received a free copy of each route. I'd be willing to do the same for DTG for the same compensation.

 

You’re wasting your time by reporting bugs. Matt even said on a recent stream that even though they are collecting feedback, they have no plans to use it.

Dovetail prioritizes deadlines over quality. They also very obviously prioritize charging their customers as much as they can, rather than doing what is right for them, as is evidenced by the existence TSW2. They clearly believe this is how they can make the most profit, despite examples of better developers such as SCS treating their customers right while also leaving Dovetail in the dust in terms of profit.

This all means that even though they are aware of bugs, they don’t bother with fixing them unless they seem like the sort of bugs that would cost them money to ignore. Dovetail somehow has enough loyal fans to continue releasing new, objectively mediocre products, without needing to worry about most of their customers getting fed up and leaving. This is very likely why they are so decidedly bad at supporting their products, while also being so willing to sell new ones.

You really want things to change? Stop buying their stuff. That includes skipping TSW2. If they see that one of their flagship products is failing, they might just address the cause of it being a failure. Either that or they will pack up and leave the industry, which would not be a big loss at this point.

 

Thank you for your answer and effort. It seems this results into a lot of discussions, but any step forward and more transparancy will be welcome.

 

I think it's pretty clear that dtg doesn't test their routes hard enough compared to a third party. And the fact that they don't bother to fix them is unacceptable. Canadian National is a prime example of dtg not testing their dlcs. Also they don't want to admit the mistakes they make compared to Rivet Games who did with the 204. This trend with dtg has to stop especially if we're paying for their products. They should have time to fix these issues or a dedicated team to fix any bugs found in their dlcs because it's ridiculous

 

They definitely do have a QA team. I know some people who are in it.

 

We know they have a QA team, the question is, what exactly are they doing? They are missing some real basic errors. If they aren't missing them and reporting them back then it appears DTG are dismissing them for whatever reason.

 

O.o

 

Well obviously they have a QA team. The obvious thing is that they aren’t given near enough time or are large enough to fix the mountain of bugs present in the game thanks to the higher ups.

Just look how many bugs have been left unfixed for TSW 1. No chance of that being fixed now. Not that there was much hope in the first place.

 

I know. My point was that they do have one, some people here don't think they do...

 

I don't think anyone actually thinks they don't have a QA team, however it's more a case of how good they are or more likely how much of what they report is actually actioned by DTG.

 

I think Plastic Pal thinks they don't.

 

And you always think yes, to what DTG says

 

Well, then you are quite a genius because that means you have created a device for scanning people's thoughts. With such a device you could obtain great power and control over other people, so I look forward to the day when you become World Master of Everything. I would really like to see the kind of world that you would build, if you were in charge. But then again, I am a fan of dystopian nightmares in literature and film... and indeed, in gaming - which is one of the reasons that I love Train Sim World.

Or..... maybe you haven't created such a device, and you are just wrong..... again. Which would be a shame, because you need to create a thought-scanner before you can start to force people to think the way that you want them to think. There are far too many people out there with the wrong kind of thoughts, and that should be corrected, don't you think? I mean... don't you think......?

To answer your concerns about a lack of a QA team at DTG...... It is profoundly clear that the QA process at DTG is not effective - and you can observe this from the lack of Quality in the final product, because the Quality has not been effectively Controlled and absolutely not Assured.

On paper there is certainly a QA team - this paperwork is required if you want to have a Right To Develop for PlayStation Compatible Products. It might even be possible that DTG has a QA team that is operational.... and... maybe... for example... they were able to find out that half of the OSD map was missing, with various marshalling yards buried under warehouses. Maybe they even said something about it to the production team... the team that placed those buildings onto the tracks. But maybe these QA experts were ignored.

End result - no effective QA process.

And yes.... I would like DTG to start thinking about making the initial plans to scope out the possibility of developing a strategy to implement the beginnings of initial ideas into commenicing a programme of tasks that could lead to a premiminary study that could recommend a future project to specify the outline model of a new approach to introducing workstreams designed to create an effective QA process as a Key Output.

That would be nice... ;-)

 

The QA team isn’t the problem imo. I don’t see anyone here saying they don’t have a QA team. I am seeing people blame them for missing obvious issues, but I personally believe a lot of those issues are being found and reported. The real problem seems to be higher ups who don’t let the developers fix reported bugs. This is especially evident with Matt admitting that they have no plans to address the feedback they are collecting, in one of the recent livestreams.

I have never seen you speak on this point. You always seem to move the goal posts around in ways that you can use to defend Dovetail with something easy and obvious, as you did here. Just once, I’d love to see you admit in even the slightest capacity that maybe Dovetail could do better.

 

I do agree about the fact that it was the management's fault about the problems not being addressed.

 

Do you also agree that if these problems are ever going to change, Dovetail needs to be given a monetary motivation to do so by people refusing to purchase their products?

Despite Sam's good intentions, even seeming to take the initiative of collecting feedback on their existing products, Dovetail appears to have no intention of changing. Just as it has been for the past decade.

 

Just to clarify - are you implying that there isn't effective management oversight of the QA process at DTG....?

As a fact.........? You are stating that.... as a fact....?

 

Yet another kindergarten food fight breaks out. What a surprise.

 

The part you don't see is what happens on the other side of the wall. I'm cautiously optimistic about changes that are underway inside the company, and while they'll take time to come to fruition - and therefore they'll take time before you notice any change in direction - if my optimism is well founded it should be seen as a significant positive.

You'll see the first signs of this with the next announcement.

 

I suppose we will see. To this end, what are Dovetail's plans when they find the need to re-use locomotives or routes they have previously released for TSW1? For example, if a new German route is released, and Dovetail finds themselves wanting to release the DB BR 182 locomotive as an addon for it. Will owners of the existing TSW1 locomotive get a free update, or will they need to repurchase the same thing a second time, but with wheel slip this time? Same with routes. If Dovetail decides to attempt NEC again, will disappointed owners of the original NEC get a free upgrade and see this route removed from the preserved collection, or will they need to purchase the same route a second time?

Let me just clarify that if we are expected to pay for the same content a second time, like we are being expected to pay for what is basically the same game a second time, but with a newer version of the Unreal engine this time, that will most certainly not convince me to return as a customer. I imagine many people would feel the same. It would only reaffirm Dovetail's current image to the public. To get a little ahead of myself here, some may say that there would be no profit in giving out V2 of something for free, but I disagree. I would not pay for it either way, but if it were a free upgrade, I may consider looking at this game again. If Dovetail shows a pattern of taking care of its customers and standing by its products, I would eventually consider supporting the company again.

This would especially hold true with my NEC example, given the low quality state of the route at present. I have long since expected that $30 addon to be brought up to an acceptable level of quality, particularly by Dovetail eliminating the bug which forces you to disable ATC in the tunnels if you want to keep to a reasonable schedule. That route needs a lot more than that, but at least fixing the ATC bug would be a start...

 

Oh, now you made me interested ( ͡° ͜ʖ ͡°)

 

Well how ironic. Maybe they should all be fired or their needs to be a overhaul because they clearly are NOT DOING THEIR JOBS. What's the point of having one if they aren't doing what they're supposed to do

 

We don't want to see tsw become the cash grab that was Flight Sim World which is why it failed in the first place. I'm pretty confident that dtg wouldn't make players pay for the same train they own for a different route cause it wouldn't be good for the company

 

Funnily enough - on PC.... DTG are making PC players pay again for exactly the same CSX:HH route.... on TSW2, even if they don't want it..... if they want to be able to access any other future TSW2 DLC that they do want.

That is why so many of them are steamingly angry.... on top of the lack of an Editor.

In TS1, the Class 66 has been sold and re-sold about ten times.

 

HST or Class 43 reskins examples.

 

Well Dovetail have proven that they don't think like you or I. Existing customers seem to matter extremely little to them. Their focus seems to be with new customers, and how much they can get away with charging them. Terrible way to do business, and yet here we are.

 

FSW failed because there are plenty of well established flight sims out there already. Dovetail couldn't compete with the other sims.

 

I think that's just business in general these days, more and more companies don't give a stuff about loyalty. Get the new customers in, get them signed up, take their money and move on to the next one. It seems it's all about quantity over quality. Turn out cheap rubbish, sell volume, people moan but because it's cheap, it's only 24.99, it's not the end of the world, so when the next cheap replacement hits the shelves the cycle repeats itself.

 

I honestly can't wait for a new train sim from Microsoft. Can you imagine a train sim with some of the tech from their new flight sim? T'would be a death blow to the dlc churning monsters out there.

 

Unfortunately, I don't see it happening, the numbers just aren't there for them to make a return on the initial investment.

According to SteamDB TSW2020 has on average 400 people playing it at any one time, Train Simulator 2020 averages 1,200.

Microsoft Flight Sim X has 2,500!

In comparison American Truck Simulator averages 4,500 and Euro Truck Simulator 2 averages 20,000!

I know consoles account for two thirds of TSW2020 users but that still only puts it at around half the numbers of FSX which is also available for xbox.

With so few customers in the train simulator genre, firstly is there any real suprise DTG are pushing that consoles must be equal and basically restricting the more creative content aspects of the PC. Secondly, to keep the money coming in from those relatively few people, the content has to keep flowing, which means getting it released in an acceptable state as opposed to near perfect so the next project and cash stream can be started.

 

Imagine where we would be if our favourite train sim had that number of users! SCS have that market (almost) to themselves, but the train sim market is fractured into several competitors. I think you're right, no-one would want to step into such a crowded field, where the customers are a fair degree more demanding than for example truck sims.

 

Euro Truck is a game, not a simulator. It's a very good game (hence the number of people playing it), but there is a reward at the destination - it's less about the journey. Now, this uses my definition of game vs simulator, but it stacks up with flight sims as well. If the journey is the reason to go on the journey, it's a sim. If the destination and associated reward, it's a game.

Also consider that maybe, more people want to fly planes than drive trains.

 

That is of course your opinion, to which you are of course entitled. However, I think you may find there many virtual Truckers disagree with you, approximately 10 times more looking at the numbers.

Euro Truck Simulator, Train Sim World? Both have Simulator in their titles.

Reward at the destination, would that not be the XP points and gold medals?