May Update Qa

Discussion in 'TSW General Discussion' started by DampWool, May 13, 2021.

  1. DampWool

    DampWool Member

    Joined:
    Sep 21, 2017
    Messages:
    31
    Likes Received:
    50
    S6 Leipzig S-Bhan service

    I find it difficult to imagine how the team managed to release the May 2021 upgrade with such an obvious defect. It tool me all of 40 seconds this morning to discover the derailment issue. It transpires that a path end point is coincident with a buffer.

    I am intrigued as to how it evaded basic testing.
    Did a team member not follow procedures ?
    Are module promotion procedures defective ?
    This is the kind of carelessness that can lead to lasting representational damage.
    There must be some very red faces today.
     
    • Like Like x 31
  2. Rudolf

    Rudolf Well-Known Member

    Joined:
    Dec 9, 2016
    Messages:
    2,497
    Likes Received:
    3,278
    I support the question, We were waiting moths to get the testing done and then you miss this type of bugs? LIRR also has some gross bugs in the pathing. It is unbelievable.
     
    • Like Like x 14
  3. dcnine#5410

    dcnine#5410 Well-Known Member

    Joined:
    Jan 5, 2021
    Messages:
    128
    Likes Received:
    337
    There are a lot of annoying bugs that I can find excusable, but every S6 service is literally unplayable! If the QA team is supposed to not care about visual things, just see if the train moves, then, well, the train doesn’t move!!! I honestly don’t get how a bug like this could get through.
     
    • Like Like x 16
  4. Jinoss17

    Jinoss17 Well-Known Member

    Joined:
    Aug 21, 2020
    Messages:
    495
    Likes Received:
    1,109
    I completely agree with you. I can forgive for the moment the old messy sounds but, S6 unplayable, LZB broken on the 182, Talent loading with the wrong settings... I'm wondering where they were looking at while testing
     
    Last edited: May 13, 2021
    • Like Like x 13
  5. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    I can easily imagine how this issue "slipped through the QA net". They don't use an effective net. Or any effective QA at all.

    And shockingly - this has happened on the PC version (and is not "only a console issue").....?

    And I suspect that this bug won't now be revisited for a number of years, as the Preservation Crew has a long and growing list of DLC to get through. Maybe they should do the bug clean-ups and patches one DLC at a time, rather than release a whole batch of reworked DLCs, all of which have unresolved problems, and/or new problems, making nearly all of the players unhappy at the same time....?

    Maybe we need a public bug tracker, so that players can accurately keep track of unresolved bugs.
     
    • Like Like x 8
  6. dcnine#5410

    dcnine#5410 Well-Known Member

    Joined:
    Jan 5, 2021
    Messages:
    128
    Likes Received:
    337
    Well luckily, they are actually working on a fix right now, Nat said that “This is already being worked and fix is in the works, you will be informed when this has been resolved”, which is good news IMO.
     
    • Like Like x 4
  7. PeteW

    PeteW Active Member

    Joined:
    Jan 3, 2020
    Messages:
    110
    Likes Received:
    86
    I guess that's why you can't complete the route in the other direction (bug reported). You get very intimate with the buffer but still have 3m to go to destination.
     
  8. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    Maybe. But this is exactly what Dmitri (IIRC) said about the game-breaking Cement Circle bug about a year ago. And the patch for that has only just arrived (this week). So maybe it will take about a year to get around to fixing this one as well....? We just don't know.

    Was a date provided for the fix - maybe they will be "working on it" for a considerable period of time.....?

    Maybe it would be a good idea for these sorts of issues to be listed in a public bug tracker, and tied to some sort of Service Standard, or target, for rectifying bugs in a timely way after they've been first reported...? This bug, like the Cement Circle bug, would fit into the "hotfix within 72 hours" target/category, given how serious it is.
     
    • Like Like x 2
  9. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    11,958
    Likes Received:
    18,333
    Well, not exactly. Cement Circle for TSW2 was fixed quite some time back. What was released last week was a fix for it in TSW2020- which officially is no longer supported at all.
     
  10. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    No. This doesn't cut any ice at all. This is why:

    - TSW 2 did not exist when the Cement Circle bug was first reported. We only had TSW 2020, so TSW 2020 must surely have been "officially" supported at that time? That bug should have been rectified immediately, a year ago, in TSW 2020, and then they would not have had to fix it twice - it would have saved DTG's time.

    But also.....

    - There is absolutely no formal or official indication from anyone at DTG that TSW 2020 is not "officially supported". Such a statement would make Sony, Microsoft and Steam very displeased. If you can find such a statement, give us a link to it. Because at the moment, DTG's bug reporting tools seem to make it quite clear that bugs in TSW can be reported - and this would be the impression that anyone from Sony, Microsoft or Steam would get, from the facilities provided for such support.


    ......but - if you were right...... then all DTG now need to do to absolve themselves from the responsibility to fix the S6 bug is to simply not fix it until TSW 3 comes out. Seems a bit unreasonable to me, but, you know, the world isn't perfect.

    Ideally, we would have a target date to fix the S6 bug. I think you gave us a benchmark in another thread... 90 days wasn't it..(?), for a bug that renders an entire DLC as "completely unplayable"? I reckon this is a 72 hour job myself, but if others are content with your 90 day suggestion, then fair enough.
     
    • Like Like x 2
  11. Clumsy Pacer

    Clumsy Pacer Well-Known Member

    Joined:
    Dec 10, 2016
    Messages:
    2,956
    Likes Received:
    3,935
    Skip to 16 minutes and 12 seconds in this:


    Sam: "Does the release of Train Sim World 2 mean no further work of any kind will be done on the content associated with the original game (route & loco add-ons, etc)?"
    Matt: "That's correct."

    It is worrying that this somehow slipped through QA. I wonder if they just decided not to bother or at least give bits of them a very quick once over and get them out because they'd been sat there for a good while - especially considering about 3 weeks ago, Sam was quoting 8 weeks before they'd come out.
     
    Last edited: May 13, 2021
  12. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    The context is important. And so is the timing (see below).

    Context
    This section of the video is not referring to bug-fixing support. It is referring to the decision to reverse the "Special Project" to get a massive number of user feature suggestions and improvements (like PIS) implemented in TSW 2020 before TSW 2 was released, and the response to this project included dozens of unconfirmed (difficult to produce) bug reports. DTG were simply overwhelmed with a vast range of requests, and so they then decided to make it clear they could not respond to everything (especially the feature requests)... but I am fairly sure that they actually fixed quite a few bugs in those last months.

    Why not play the video from minute 74.... (1hr:13m:00s), where they talk about clarifying how they intended to handle bug reporting (which sort of became the Roadmap...) - here they talk about communication - making it clear what is considered a bug, and what is a designed feature (the classic non-bug stopped at a red light issue), and the need to improve communication on bug resolution - and at the time, this was in reference to TSW 2020 - they didn't say that only TSW 2 would get future patches.

    But if you can find a written statement (that can't be multiply understood, or re-misconstrued), that says (now), that TSW 2020 is not offically supported, let me know. Because it is on sale - and there is no indication that it isn't supported. - quite the reverse, in fact.


    Timing (re: Cement Circle):
    And then check out the video from 1:21:00..... here you will discover that the Class 20 is due to come out soon - and at no point did either presenter say anything like:

    "You know, we still have a Class 20 DLC to release under TSW 2020, but since we have decided to not do any more work on TSW 2020, as we discussed earlier, if the new DLC is bugged, or if it breaks another DLC, like the Class 31 DLC's primary trophy supported content - then - you know, we will not be fixing it because we have already decided that the DLC falls under the new regime of "not doing any more work on TSW 2020", as discussed earlier....."

    They didn't say that. So, Cement Circle should have been fixed, immediately after it became broken by the release of the Class 20 DLC (with its No.1 end/nose issue, that really mucks about with previously coded double-headed trains).
     
    • Like Like x 1
  13. OldVern

    OldVern Well-Known Member

    Joined:
    Nov 2, 2020
    Messages:
    12,357
    Likes Received:
    24,827
    Probably driving through Colorado in ATS2 and wondering, "Wow, how come we can't get this sort of experience in TSW..."
     
    • Like Like x 7
  14. pacificorca#1435

    pacificorca#1435 Active Member

    Joined:
    Apr 9, 2021
    Messages:
    60
    Likes Received:
    117
    It is a bit concerning that QA tests don't seem to be catching bugs and issues that really should be obvious. I might have been able to cut them some slack since the preservation updates are a relatively new thing and the testing process might run a bit different. Maybe the testers were given a different build that didn't have the S6 bug? But combined with things like the weight issues on Arosa and Cut-out on Clinchfield it's probably time to think about changing the testing process.
     
    • Like Like x 7
  15. Cramnor

    Cramnor Well-Known Member

    Joined:
    Aug 21, 2020
    Messages:
    1,066
    Likes Received:
    2,138
    QA seems to be not responsible for identifying immersion breakers, now it looks like they are not responsible for identifying game breakers, so what are they actually responsible for at DTG? Delaying releases? Seems a weird concept imo, that time could probably be invested better :) The signalling system(s) for example, they would really benefit from some fixes :)
     
    • Like Like x 5
  16. Dinosbacsi

    Dinosbacsi Well-Known Member

    Joined:
    Sep 20, 2017
    Messages:
    3,316
    Likes Received:
    8,684
    At this point I'm starting to believe QA for them consists of "Does it build? Nice. Does it start up without crashing? Nice, release it."
     
    • Like Like x 5
  17. Monder

    Monder Well-Known Member

    Joined:
    Aug 14, 2019
    Messages:
    2,490
    Likes Received:
    7,663
    We've had something like this in the Roadmap... until they took it out... I wonder why.
     
    • Like Like x 6
  18. Dinosbacsi

    Dinosbacsi Well-Known Member

    Joined:
    Sep 20, 2017
    Messages:
    3,316
    Likes Received:
    8,684
    We already have the perfect way to keep track of unresolved bugs. Just play the game and you will entounter all of them in 1 hour of gameplay.
     
    • Like Like x 6
  19. breblimator

    breblimator Guest

    What happened during the last update and HHL - DTG, a shame.
     
    • Like Like x 4
  20. fabristunt

    fabristunt Well-Known Member

    Joined:
    Dec 23, 2019
    Messages:
    637
    Likes Received:
    1,220
    I would also like to receive an explaination for what happened. It's not the first time and won't be the last. Come on DTG! You have no shame...
     
    • Like Like x 3
  21. Lamplight

    Lamplight Well-Known Member

    Joined:
    Aug 20, 2020
    Messages:
    3,782
    Likes Received:
    6,314
    I hope we‘ll get some answers next Tuesday. This patch was a real mess if we’re being honest and I hope it‘s not indicative of what we can expect from Adam‘s team in the future.
     
    • Like Like x 6
  22. mancunian#7861

    mancunian#7861 Active Member

    Joined:
    Apr 9, 2021
    Messages:
    234
    Likes Received:
    109
    Personally I don't understand why DTG don't create a topic for bugs on this forum rather than having to go off forum to another website to report bugs. Wouldn't it be easier, quicker and better to have it right here on this forum? Could have a section for known bugs, bugs being fixed, report a bug etc then we could all see at a glance if a problem we've encountered is known about or is being fixed etc. It could even have a 'me too' button so they can see how many are experiencing the same problem and on which platform/DLC etc. Or am I just being naive thinking that perhaps "sometimes the easy solutions are the best" doesn't apply?
     
    • Like Like x 6
  23. LeadCatcher

    LeadCatcher Well-Known Member

    Joined:
    Dec 9, 2016
    Messages:
    1,426
    Likes Received:
    3,764
    While I agree that the number of issues that have been reported are disturbing and disconcerting, they are at least doing bug fixes, which is a rather new concept for them compared to TSClassic :) .
     
    • Like Like x 4
  24. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    The reason why the "bug-list" bit was removed from the Roadmap, as I understand it, was that they found it difficult, internally, to keep the list updated (and I believe them). It also wasn't accurate, as it wasn't clear what the bugs were to everyone (internally)..... I guess that they did not tag the line items to individual team members, so no ownership, and no responsibility for them. But this is not because of inherent problem properties that come from "lists" - it is simply because the list-management process was/is incredibly disorganised.

    Apparently it was decided, for a reason that only the Good Google can know, that it was a good idea to let the list be managed by Communitying persons, as a pure communication tool. Communitying people are normally hired for their communication skills, and project/programme management is not usually part of their skill set - so it is utterly bizarre to give them any sort of project management role - and this sort of initiative is doomed to fail because project management (and list management) requires specific skills that can be quite hard to come by in an organisation that doesn't put a very strong focus on competent project management in their product development. Although I suspect that the pantry is always well stocked.

    Anyway - I actually think that this sort of granular "bug-list" detail should not be in a Roadmap - because a Roadmap is normally a strategic forward look style of document, with loose timescales. I think that some people in DTG think that "Roadmap" just means "a list of things with loose timescales" - so at home you might have a really poor idea for a "Roadmap", for example, to "eventually do some DIY in the kitchen", but fully intending to leave the power drill nice and clean in its wrapping, in the box under the stairs. But a proper Kitchen Roadmap would indicate what you want your kitchen to do in the future (say, you want to have Nigella Lawson host a dinner party for twenty, in three years time, for your wedding anniversary) - and it would include steps you need to take to get the result that you want (as broad stages), given the likely expansive facilities that Nigella would require to complete that task, with the detail to be filled in as you pass through each stage. This could include a regular review of Nigella's latest cooking programmes on TV.


    The TSW Roadmap, in trimmed down form, indicating loose plans for future releases, is a better document without bug-tracking. It still isn't a Roadmap though - it's just a list. It doesn't have any "Map" bits in it.

    If you want to add "strategy" and "project management" to your CV, then you need to do a little bit more than create a special project that has a "list of things" with "loose timescales".


    What we need is an open bug tracking list that lists reported bugs that are reported by customers, important to the customers, understood by customers, and meets the needs of customers, including specific timescales from first report date, to resolution - with target timescales for each bug to be closed out.
     
    • Like Like x 4
  25. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    Brilliantly said.

    And, I admit, with fewer words than me - although I do like Nigella's cooking.
     
    • Like Like x 2
  26. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    11,958
    Likes Received:
    18,333
    Which could be implemented as a read-only Google Docs spreadsheet, accessible from this site.
     
    • Like Like x 3
  27. mancunian#7861

    mancunian#7861 Active Member

    Joined:
    Apr 9, 2021
    Messages:
    234
    Likes Received:
    109
    Thank you for saying so .. although I've never tasted Nigella's cooking personally. I've got fish and chips tonight though. :D
     
    • Like Like x 1
  28. pacificorca#1435

    pacificorca#1435 Active Member

    Joined:
    Apr 9, 2021
    Messages:
    60
    Likes Received:
    117
    One option for bug tracking I've seen work particularly well is the combination of Jira and Trello for back end tracking and public facing tracking of bug fixes. Obviously the feasibility of this depends on how invested DTG is with whatever bug-tracking system they're currently using. It would almost certainly involve some relearning for the devs but would pay dividends later on, might be worth a try after Rush Hour.
     
    • Like Like x 1
  29. Monder

    Monder Well-Known Member

    Joined:
    Aug 14, 2019
    Messages:
    2,490
    Likes Received:
    7,663
    That's why a clear roadmap on the main site with a bug list on forums was in my opinion a pretty good compromise. The main juicy stuff is in the article, the tracking on the forums. Yet even that has been dropped.
     
  30. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    11,958
    Likes Received:
    18,333
    I don't care if her cooking tastes like spoiled Marmite- I'd simply enjoy watching her cook
     
    • Like Like x 2
  31. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    I think that the only way to ensure that a Community Bug Tracker is implemented, is for the Community to do it.

    Thinking more about the suggestion made by mancunian#7861..... a key purpose would be to provide a "sanity check" for players that first encounter a bug, so that they can check to see whether a problem really is the result of a bug, and if so, when it was reported, how long it has remained unresolved and what workarounds might exist until the patch is made.

    A key thing for me is to prevent any really serious bug becoming "historic" - say, over a year old... because when such bugs are not recognised, or the reports are ignored, there is a risk that they will never ever be fixed. But the DLCs, of course, will always be on sale - so new players will trip up on them.
     
    • Like Like x 4
  32. mancunian#7861

    mancunian#7861 Active Member

    Joined:
    Apr 9, 2021
    Messages:
    234
    Likes Received:
    109
    Precisely Angel. I couldn't agree more. A "sanity check" is an excellent way of putting it, too. A win-win one would have thought!
     
    • Like Like x 1
  33. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    11,958
    Likes Received:
    18,333
    The only problem with this approach is that the community have no common definition or criteria for what constitutes a "bug." Myself, I wouldn't include deliberate design decisions, even when they are bad ones, like the low-door Dostos. Nor should the list, I think, include the more trivial sorts of complaints of which there is no shortage. To merit inclusion, the bug should be 1) accidental and 2) adversely affect gameplay. But how would any set of qualifiers be enforced, and the database not be cluttered up with "wrong breed of grass" and "wrong size brake lever" sort of complaints?
     
    Last edited: May 15, 2021
    • Like Like x 2
  34. mancunian#7861

    mancunian#7861 Active Member

    Joined:
    Apr 9, 2021
    Messages:
    234
    Likes Received:
    109
    Yes, good points, Bill. There are pros and cons to every solution I suppose.
     
  35. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    Well, the idea of a bug tracker has been proposed before, and some of these issues have been fleshed out and discussed in some detail, with supporting arguments and bug criteria, as follows:

    -------------------------------
    Initial proposal for an off-site JIRA+++ style tool managed by DTG (by Rudolf):
    https://forums.dovetailgames.com/threads/a-public-bug-tracker-would-benefit-us-all-poll.15931/

    A Forum based tracker, managed by DTG (maybe this became the intial version of the Roadmap...?):
    https://forums.dovetailgames.com/threads/forum-list-of-known-bugs-and-issues-poll.18278/

    A Forum based tracker, managed by the Community (which is what we probably need now) :
    https://forums.dovetailgames.com/th...ugs-issues-tracker-for-train-sim-world.23410/
    -------------------------------

    All three proposals carried over 90% support in the polls - and my one, the Forum based tracker managed by DTG, has/had 100% support (to date) . But I no longer think that a bug tracker managed by DTG would be dependable, especially when we see issues like the Bakerloo Line MR fault just being dropped from the "Roadmap", when that is a viciously vicious super-vicious vicious bug in "a base route" for the TSW 2 platform that everyone is required to own, impacting 30% of players (by driving style, across all platforms). But we also need to track issues like the BR 101 PZB lights (apparently now resolved, but resurfacing in the HBL BR 182.....?), which seem minor, but are actually really very significant for involved simulator gameplay.

    But.... if a list is to be managed by the Community, then a few people would need to agree the process and format for the lists, and then step up, and build, and maintain it. That cannot be just one person, not now, with the sheer number of routes in the back catalogue, and because just one person might one day just run out of electricity.... so to speak. ;-)
     
    Last edited: May 15, 2021
    • Like Like x 2
  36. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    11,958
    Likes Received:
    18,333
    While that's true, only a DTG-managed database would include the important information what DTG is doing or is planning to do about it.
     
    • Like Like x 2
  37. synthetic.angel

    synthetic.angel Well-Known Member

    Joined:
    Aug 27, 2019
    Messages:
    1,484
    Likes Received:
    2,265
    Yep - and that would be entirely for DTG to know. Community members would only need to know (from the list):

    ------------------------
    - Is that thing I experienced really a bug, has anyone else encountered it....?
    - Oh, it was reported on the Forum, discussed and confirmed as a serious bug, and is on a list, with a report date, so I am (probably) not going loopy, after all......
    - Oh, it was reported 18 months ago... that's not good... why hasn't it been fixed...(?) I'd like it to be fixed...!
    ------------------------

    If DTG give feedback, such as "we cannot reproduce", then people can try and reproduce it (I would prefer that the list only contained bugs reported and seconded by another Forum member).

    DTG could give feedback like "we know it's a bug, and we are not fixing it" (as they initially tried to do with TSW 2020 Cement Circle.... with all that discussion on maybe getting back to us with a green light or not, for a patch....).

    But, most importantly, the list would only contain bugs that are important to the Community......

    And then......

    .....people can make purchasing decisions based on how much they are prepared to pay for a product that is known to be broken, and to what extent it is broken, which would provide an incentive for DTG to crack on get things fixed... and maybe.... maybe...... do some QA on their releases before publication, resulting in:

    More QA = More Money
     
    • Like Like x 2
  38. ARuscoe

    ARuscoe Well-Known Member

    Joined:
    Apr 9, 2018
    Messages:
    10,832
    Likes Received:
    10,483
    I'm sure we all know some people who would love to go bug chasing... so long as there was a result at the end (ie a fix)
     
    • Like Like x 4
  39. xblackwolf90

    xblackwolf90 Well-Known Member

    Joined:
    Jan 18, 2020
    Messages:
    380
    Likes Received:
    496
    I was really excited for the updates, particularly for RT as this was one of my favourite routes in TSW2020. The poor audio unfortunately meant I didn't play it much in TSW2.

    Whilst the PIS implementation is great, and other improvements to the physical route are good, I really struggle to understand how the bigger issues got through testing.

    Rare or difficult to reproduce bugs (I.e. The instance where Sam derailed the TGV at high speed on stream) are understandable. But, if a single tester ran an S6 service just once, they'd find it can't be completed.

    The big issues for me in the RT update that I can't understand how they got through testing are:

    1) Broken S6 services.
    2) No running audio in the Talent 2.
    3) Broken LZB 'Ende' function in the 182.
    4) Broken PZB, LZB and dial lights depending on the format.

    I would much rather have waited another month or even longer to get an RT update with these issues fixed.
     
    • Like Like x 5

Share This Page