Open Letter To Dtg Part Two (qa Issues)

Discussion in 'TSW General Discussion' started by tsw2, May 31, 2022.

  1. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    Oh I see. Maybe the devs that went there simply wanted to pursue other video gaming development opportunities. It's quite common in the video game industry for devs to leave certain studios and go to others to find other opportunities. So devs leaving DTG certainly isn't a bad thing in my opinion unless they had problems with DTG.
     
    • Like Like x 1
  2. IsambardKingdomBrunel

    IsambardKingdomBrunel Well-Known Member

    Joined:
    Oct 6, 2017
    Messages:
    515
    Likes Received:
    941
    Worms now that was a good game, better than TSW2 :) :) ;)
     
    • Like Like x 3
  3. dhekelian

    dhekelian Well-Known Member

    Joined:
    Nov 24, 2020
    Messages:
    2,076
    Likes Received:
    2,703
    I think they were from the QA team but can't remember what Sam said now. My thinking was if they had indeed left the QA team perhaps there was a problem and then with Matt saying they were hiring people I found that odd although I accept there could be a totally innocent reason.
     
    Last edited: Jun 6, 2022
    • Like Like x 2
  4. Richard CZE

    Richard CZE Well-Known Member

    Joined:
    Aug 20, 2020
    Messages:
    694
    Likes Received:
    770
    There is always an error in the code. I don't mind. It bothers me that I have to wait six months or more to fix the bug.
     
    • Like Like x 6
  5. March Hare

    March Hare Well-Known Member

    Joined:
    Apr 22, 2022
    Messages:
    549
    Likes Received:
    1,022
    Five plus years in the case of the save game core feature that should have been fully working at launch or patched very shortly afterwards, and development documentation for internal and third-party development that ensures all add-ons meet the same requirements for save functionality to, well, function.
     
    • Like Like x 5
  6. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    The save game bug is a very complex issue. It seems it may have to be fixed as a route by route basis because of the signaling issues it causes after loading a save
     
    • Like Like x 2
  7. tsw2

    tsw2 Well-Known Member

    Joined:
    May 31, 2022
    Messages:
    964
    Likes Received:
    1,935
    I also wonder why we can't have different season in the scenario creator.
    I mean this is another one of these odd TSW bugs or issues. I never heard an official explanation for that.
     
    • Like Like x 2
  8. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    There was no explanation for that which I found odd. Did they not think that players would like to change their weather settings
     
    • Like Like x 2
  9. tsw2

    tsw2 Well-Known Member

    Joined:
    May 31, 2022
    Messages:
    964
    Likes Received:
    1,935
    Most likely another case were they started with a bad code base and now it would be much work to change that since they had to do the whole thing from scratch basically. But these errors should not happen.
     
    • Like Like x 3
  10. VictoryWorks

    VictoryWorks Well-Known Member

    Joined:
    Sep 22, 2017
    Messages:
    263
    Likes Received:
    1,379
    This video about QA testing on Cyberpunk 2077 is well worth a watch.. and if you pause at 10:55 and look at the client list it might explain a great deal
     
    • Like Like x 4
    • Helpful Helpful x 2
  11. Purno

    Purno Well-Known Member

    Joined:
    Jan 16, 2018
    Messages:
    3,043
    Likes Received:
    4,967
    That's an interesting video VictoryWorks . I hope DTG themselves are aware of the problems with the QA company that's apparently working for them.
     
    • Like Like x 2
  12. tsw2

    tsw2 Well-Known Member

    Joined:
    May 31, 2022
    Messages:
    964
    Likes Received:
    1,935
    I guess they do the work cheaper then others...
     
    • Like Like x 2
  13. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    Oh my. I am just lost for words. That is crazy. The fact that this doesn't only happen with Cyberpunk but with other games as well could be a contributing factor as to why TSW has had so many issues like the crashing that happened on London Commuter. From streams it didn't even seem DTG were made aware of the crashing until the prodcut was released.

    How in the world does a QA company hires people that have no experience doing qa not give them enough training time to get to grips with the product. When DTG say that their QA doesn't really understand Train Simulation or isn't familiar, quantic labs not giving them enough time to get familiar could be a contributing factor as to why.

    The fact that employees can't speak up about issues with fear that they may have to resign is also ridiculous. If there are problems in a work space then employees should be allowed to speak up if their is a problem.

    Also QA having to meet quotas probably explains why bugs for this game seem to slip out alot. This company should give them more time to test and fix issues. From a consumer standpoint it is better to have a well functioning product that doesn't release with a ton of issues even if it means it was delayed rather than a product that releases riddled with bugs that makes the product non-enjoyable. When it was announced that Test Drive: Solar Crown would be delayed to next year, from what I've seen, there seems to be a positive feeling from the community about that since in theory the game when it releases will be a stronger/better state to if it had released this year. There are many players in the video game industry that don't mind a game being delayed if it means the product will be better in the end.

    Upper Echlon Gamers is a pretty good youtuber. The fact that his information also came from a whistleblower at Quantic Labs means that DTG can't come and say that he doesn't know what he is talking about when he mentions that these practices happen with the other games that Quantic Lab work on. If I was DTG and if these practices are indeed true which seems like the case, then frankly it wouldn't make sense to do business with this company. Their working practices seems to be a contributing factor as to the issues that we have seen with TSW2. The fact that these practices still happen to this day as well (this video came out two days ago) is very concerning and I hope DTG are aware of the problems of the company that works for them.
     
    Last edited: Jun 27, 2022
    • Like Like x 1
  14. OldVern

    OldVern Well-Known Member

    Joined:
    Nov 2, 2020
    Messages:
    18,087
    Likes Received:
    36,773
    Interesting video clip. You would have thought items like the flying cars and spinning motorcycles would have been picked up by the devs playtesting themselves. Or maybe they don't and if a similar situation pertains with DTG not surprising stuff slips through.

    The only slight confusion here is that the clip states the company is based in Romania, but I'm sure it's been mentioned DTG are using India based QA for TSW etc. testing, unless more than one organization is involved.
     
    • Like Like x 2
  15. tsw2

    tsw2 Well-Known Member

    Joined:
    May 31, 2022
    Messages:
    964
    Likes Received:
    1,935
    Did they really say that?
    But it does not matter either way.

    In the end, DTG is the company that sells TSW and they are resposible for the (lack of) QA.
    If they sense that the company they hired to do the QA does not perform as expected they could fix this easily. If they stay with whatever company they hired for QA despite repeated and obviouse issues then i would not blame the QA companie, i would blame DTG because im a customer of DTG. They sell TSW2 and they are the developers. If i hire someone to build a cabin and this guy hires another companie to do the roof and later i would have issues with the roof, i would complain to the guy i hired and demand that he fix the issues.

    So lets not shift blame to the external QA. Nobody forces DTG to use external QA and nobody forces them to stay with a company that does not deliver good results.
     
    Last edited: Jun 27, 2022
    • Like Like x 2
  16. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    They have made similar remarks to that in the past. There is reason why I used "could" or "probably" in what I said. My intention isn't to just blame one party only. As you said DTG are ultimately the ones that sell the product. The buck stops with them but I was pointing out that there could (key word is could) be other factors that explain why a product releases the way it does. QA could be one of those factors
     
    Last edited: Jun 27, 2022
    • Like Like x 1
  17. Calidore266

    Calidore266 Well-Known Member

    Joined:
    Jun 17, 2021
    Messages:
    1,499
    Likes Received:
    2,206
    I'm pretty sure I also remember something along those lines being said as part of a QA discussion on a stream, where DTG also acknowledged the need for improvements in the QA process.

    It's worth mentioning that even so, DTG has never put the final responsibility for the game on anyone but themselves. They acknowledge their missteps and failures, often in detail regarding what happened and what they hope to change.

    Also worth mentioning is that we have no idea how much, if any, work, Quantic Labs has actually done for DTG. The simple presence of a logo means nothing. A while back, someone posted a host of logos they'd found in TSW's software, many of which aren't used in the game.
     
  18. Calidore266

    Calidore266 Well-Known Member

    Joined:
    Jun 17, 2021
    Messages:
    1,499
    Likes Received:
    2,206
    Upper Echelon has released a follow-up video, btw.

     
  19. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    12,897
    Likes Received:
    19,698
    Sometimes the lowest bidder isn't the best deal.
     
    • Like Like x 3
  20. Calidore266

    Calidore266 Well-Known Member

    Joined:
    Jun 17, 2021
    Messages:
    1,499
    Likes Received:
    2,206
    Sometimes? :D
     
    • Like Like x 1
  21. Purno

    Purno Well-Known Member

    Joined:
    Jan 16, 2018
    Messages:
    3,043
    Likes Received:
    4,967
    Yes, but I think any company can expect any contractors to abide the laws. If DTG has been lied to by their QA partner, which, AFAIK, is considered illegal, DTG is as much a victim as we are, even though they remain responsible the final quality of their product.

    Sometimes more expensive alternatives aren't always a better deal.
     
  22. OldVern

    OldVern Well-Known Member

    Joined:
    Nov 2, 2020
    Messages:
    18,087
    Likes Received:
    36,773
    One other thing that occurs to me from the information above is, how good really is an outsourced company at testing something as technically and procedurally complex and unique as a train sim? Is an employee one day testing a FPS/RPG really going to understand the finer points of UK, German (or anywhere else) signalling? Do they understand how locomotives perform attached to different loads, or the differences between freight and passenger operation. This is a very specialised niche and it really needs people with at least an inkling of the subject matter.
     
    • Like Like x 8
  23. Purno

    Purno Well-Known Member

    Joined:
    Jan 16, 2018
    Messages:
    3,043
    Likes Received:
    4,967
    I imagine the QA company receives a list of things they need to test for.

    And one doesn't really need to know much about signalling systems and locomotive performance to discover sounds that aren't audible, timetables getting stuck, AI trains running a red light, passengers getting stuck, etc.
     
    • Like Like x 1
  24. pinxtonpaws

    pinxtonpaws Well-Known Member

    Joined:
    May 24, 2022
    Messages:
    460
    Likes Received:
    1,200
    This is why I can't for the life of me fathom out why they don't just release the addons into early access. Sure, there will be always be a bunch of gamers who just get it because it's cheap and undoubtedly a group who will still only buy when it's fixed/on sale but judging by some of the comments within these forums there are also a lot of very knowledgeable people who seem more than prepared to lend a hand. It just strikes me that, if only for credibility's sake, addressing a problem prior to launch with people that know their stuff makes far more sense than addressing the exact same problem when it's there for everyone to see.

    I don't know, I just sometimes get the impression that DTG think they're too big a company to concede defeat and take the early access route.
     
    Last edited: Jun 28, 2022
    • Like Like x 2
  25. roysto25

    roysto25 Well-Known Member

    Joined:
    May 1, 2019
    Messages:
    315
    Likes Received:
    357
    These issues with QA are unfortunately endemic with outsourced work and absent DTG going against the industry trends will not be easily resolved. In the case of India, I have extensive experience working with and managing outsource contracts. Personnel are usually competent, but tend to be recent graduates of IIT with 'out of the box' knowledge of software suites and not extensive working experience. Also, burn-out is very common - cost of housing in Mumbai, for instance, is prohibitive for low-wage employees, so commute times of 3 hours each way are common, leading to very high staff turnover. So typically you lose what I call 'group memory', i.e. no-one is around who remembers the solution or approach for a problem which was encountered before and 'out-of-the-box' thinking is rare. Add to that the cultural biases - you can be told what you want to hear rather than cold hard truth. The reality, in the UK as with USA - I am sure elsewhere too - is that companies and the country as a whole have been unwilling to pay for widespread technical training and will not commit to the cost of in-house development of talent, which inevitably means a period of relative low productivity for trainees. The result is a huge transfer of knowledge to our current and future competitors and incidentally also produces a corporate need of imported skills. I know of US companies where 90% of the IT department is Indian on various visas. A major source of Hewlett Packard and IBM income these days is contracting these employees to other companies. As a side note - Japan leads the world in micro motors. Their companies outsource basic product assembly to other countries, but never allow micro motor assembly or production out of the hands of Japanese technicians. There will be a secure area within the building restricted to those employees. Result is generally high QA competence.
     
    • Like Like x 4
  26. Pipe

    Pipe Well-Known Member

    Joined:
    Jan 11, 2022
    Messages:
    2,083
    Likes Received:
    4,352
    That´s very well analyzed and it meets also my experiences, which are similar.
     
  27. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    12,897
    Likes Received:
    19,698
    I do not recall DTG ever saying their QA was outsourced to India. What they did say was that 1) they use outsourced QA and 2) that it had been hit pretty hard by Covid. That led lots of people on the forum to assume "India;" but it could just as well be Romania.
     
    • Like Like x 1
  28. Tank621

    Tank621 Well-Known Member

    Joined:
    Sep 13, 2020
    Messages:
    840
    Likes Received:
    1,515
    In the back of my mind, I think I recall them saying something about using an Indian company but that was months ago and I can't remember exactly what they said.
     
  29. dhekelian

    dhekelian Well-Known Member

    Joined:
    Nov 24, 2020
    Messages:
    2,076
    Likes Received:
    2,703
    I could of sworn Matt said a little while ago now it was outsourced to 'India'. Could be wrong but sure he did.
     
  30. VictoryWorks

    VictoryWorks Well-Known Member

    Joined:
    Sep 22, 2017
    Messages:
    263
    Likes Received:
    1,379
    I really hope DTG don't still use Quantic Lab - a lot more is coming out about this story

     
    • Like Like x 2
  31. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    12,897
    Likes Received:
    19,698
    Is DTG going to comment, I wonder?
     
    • Like Like x 2
  32. solicitr

    solicitr Well-Known Member

    Joined:
    Jun 20, 2020
    Messages:
    12,897
    Likes Received:
    19,698
    Well, DTG certainly appears on Quantic's client list.
     
  33. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    Well according to Quantic Lab's client list, Dovetail Games is there so you don't need any more conformation than that really. Granted, they may not work on TSW though
     
    • Like Like x 1
  34. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    After watching this video, there are some even more odd things this company does.

    For instance they outsource work that was already outsourced to them. They did this with Project Wreckfest when they outsourced testing to India because they lied about the number of workers they actually had. I wouldn't be surprised if they outsourced Dovetail's work to India. It's baffling that they just outsource work given to them to other parties. Shouldn't that be up to the client themselves?

    They apparently ask workers that they paid in overtime to give back the money they paid them

    They advertise things that are false such as lying about the number of native and fluent language speakers they have. When translating video games from one language to another, it's important to have as many people who are fluent in that language as possible to translate a game perfectly. Quantic apparently would force these employees in charge of translating to use unreliable third party websites due to lack of experience. This leads to poor and unorganized results

    New Employees that had no experience would allegedly be given important tasks because workers for this company would just quit in a short period of time. That certainly isn't a good thing and I could just go on and on. Honestly this company seems to have a out of order work place.

    Outsourced QA is advertised as a good thing that will delver good results but these videos show the problems with outsourcing QA and QA in the video game industry in general. There is too much malpractice. Employees are mistreated, can be abused and are underpaid. New workers work in a workspace in which the employers don't care about them nor the quality that they produce. Inexperienced workers are given tasks that they frankly shouldn't be given which leads to poor results. Executives of these QA companies seem to care more about the bottom line rather than the quality of their clients product. Heck, there are employees at different companies that apparently discussed facing similar issues. This is the end result and while yes it is ultimately down to the video game company to deliver a good quality product, it isn't fair for a video game company to be given promises by a QA company and false assurances which can contribute to a messy product which hurts the consumer in the end.

    QA workers don't deserve to be mistreated or underpaid. They deserve better treatment because they are the ones that give feedback so that consumers get a product that isn't extremely buggy and is riddled with issues when said product releases. Sure while this doesn't happen at every qa company or team, this is a widespread problem that needs to be addressed and I am glad Quantic Lab got exposed and these qa topics are being discussed by many people in the video game community.

    If Dovetail Games are doing business with this company, then they should not be doing any business with this company with any of their games in my opinion.
     
    Last edited: Jun 30, 2022
    • Like Like x 3
  35. OldVern

    OldVern Well-Known Member

    Joined:
    Nov 2, 2020
    Messages:
    18,087
    Likes Received:
    36,773
    There’s a certain irony that a company which prides (no pun intended) itself on diversity, inclusivity and all the baggage that goes with it, then has no scruples in dealing with companies that treat their employees so badly.
     
    • Like Like x 7
  36. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    I guess some companies love being hypocrites.

    That or they may have not known whilst doing business. It can be easy to mislead people these days
     
    Last edited: Jun 30, 2022
    • Like Like x 1
  37. Purno

    Purno Well-Known Member

    Joined:
    Jan 16, 2018
    Messages:
    3,043
    Likes Received:
    4,967
    I wonder if DTGs current focus on bugfixing, rather than developing new content, is related to the flawed outsourced Q&A...
     
    • Like Like x 3
  38. Pipe

    Pipe Well-Known Member

    Joined:
    Jan 11, 2022
    Messages:
    2,083
    Likes Received:
    4,352
    I fear you´re quite on spot here. This oh-so-wonderful globalization thingy DOES have its downsides. We built some traps only to walk into them ourselves. See dependencies on China or Russia.

    But we need to take our time for political correctness: You don´t call a lie a lie! You call it "economical with the truth". There! :D
     
    Last edited: Jun 30, 2022
    • Like Like x 2
  39. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    I forgot about this lol.
     
    • Like Like x 1
  40. DTG Alex

    DTG Alex Senior Community Manager Staff Member

    Joined:
    Jun 16, 2022
    Messages:
    1,576
    Likes Received:
    13,269
    Hey everyone, I just wanted to jump in myself and clarify a few things.

    I've checked with the team, and we currently use Quantic Labs for our localised (translation) QA, so not for 'gameplay', which is predominantly done in-house.
    We'd usually use outsourced QA when there's a skillset or need we don't immediately have in-house ('stress' testing new builds, checking translations, etc.).

    We've discussed on a couple of Roadmap streams now, but there has been a change how we do QA internally - new leadership, new processes, etc.. We've now introduced our Xbox beta team as well to help us with feedback at an earlier stage. Both things mentioned above have been instrumental in the past couple of patch releases (GWE/NTP/WSR Preservation Crew, Xbox Jackhammer, etc.).

    There's always more we can do, but we're hopeful you're seeing some positive progress.
    We're always trying to keep improving, thank you for the feedback.
     
    • Like Like x 11
    • Helpful Helpful x 1
  41. Calidore266

    Calidore266 Well-Known Member

    Joined:
    Jun 17, 2021
    Messages:
    1,499
    Likes Received:
    2,206
    Most likely QL didn't put this information in the contract.

    And thanks to Alex for the input.
     
    • Like Like x 1
  42. VictoryWorks

    VictoryWorks Well-Known Member

    Joined:
    Sep 22, 2017
    Messages:
    263
    Likes Received:
    1,379
    Thanks for the reply Alex that is good to hear. Although might I suggest that you watch the section in the previous video relating to QL's translation services.
     
    • Like Like x 3
  43. tallboy7648

    tallboy7648 Well-Known Member

    Joined:
    Apr 9, 2020
    Messages:
    6,567
    Likes Received:
    10,808
    Thanks for the reply. Although I think the way QL do translation services suggests that doing business with them perhaps isn't a good idea
     
    • Like Like x 2

Share This Page