Tutorial Lidar Tutorial

I'm waiting for matts tutorial every time i try get the area I want and import it I get a flat surface

 

Matt also said that he unfortunately found a bug in the LiDAR import, while prepping his tutorial.
he is going to get it fixed.
Its also a good thing in a way, because it means He and Us, will be able to get a better understanding as to how it all works properly.

 

Struggling like many of you. Got the LIDAR data, merge it, Pseudo M it, tile it, export it, add it to the UE project. hgt data works fine, apply the LIDAR asc's and it just modifies the tile to be flat and about 1km up in the air.
Tried it every way I can think of and just can't get the LIDAR to work correctly. Wondering if I fouled up an import setting when I was initially messing around with no clue.
Is there any way to set the entire Editor back to default settings?

 

I have imported UK LIDAR data and got it to work using Lulon's method although I had to reinstall the editor at one point as it kept crashing.
I am trying to transfer my Train Simulator (Classic) route based around Chippenham Wilts and the LIDAR data has loaded in exactly on the spot I chose as the route origin. Track bed, embankments, roads etc are all visible and there are even flat "pads" where large buildings are in real life.

 

I had the same problem.
But then I re-read the original explanation at the top of this thread. And there was one point that I had overlooked:
point 7: It is very important to get the data lined up with the correct coordinates. i.e. that the merged tif files must be adapted to the coordinate system used in TWS4 in general because it is: EPSG:3857 - WGS 84 / Pseudo-Mercator.
Then I think you will be successful too! Good luck.
By the way: There is a batch button (at the bottom) which works perfectly when tiles need to be converted from .tif to .asc - and then it's no problem with the many tiles

 

So, I've had a crack at this using a combination of the tutorial above and the video. It goes kind of well in QGIS, but I having trouble understanding the correct coordinate systems to use, because I'm using a UK route area and the tutorials are working from Australia/German areas I think.
Could anyone who's had success with the UK lidar just highlight what coordinates systems they used - becuase when I remimported my TIFF files to check, they spawned on the equator! Obviously they won't show correctly in game then!

 

Try EPSG:25830 (UTM Zone 30 North)

 

For all the guys that were getting flat terrain upon importing LIDAR (and in the air) , I think I know the reason - Tick the Use Pseudo Mercator Box in the snap to landscape tab (see below)!
I was getting the flat in the air terrain, this solved it. Hurrah.
It was quite a tedious process, but a combination of this post and the need for the correct coordinates (and the video linked) worked well. Took long enough though. Thanks to all that helped.

 

Got it sorted today, thanks to everyone for the detailed comments and instructions

 

Will give this a g

you have to use the tif ones. in the dropdown menus choose LIDAR Composite DTM, year 2022 and resolution DTM 1M

 

problem i'm having when i select apply lidar data to selected lanscape proxys i get a solid block pop up (see image). I also uninstalled and reinstalled the route editor then uninstalled installed qgis and installed the 3.28."firnze" version this seems to have sorted some issues and my editor doesn't crash, well it hasn't up to now.

EDIT: Just a quick thought. when i went to created a new route the area I chose is at a sea level of approximately 19m but I left this at 0 could this be the reason I'm seeing blocks when I apply lidar data to selected landscape proxys? And when I go into network edit mode my track laying circle is up in the air

 

LIDAR composite DTM is the one i downloaded select the latest date in next dropdown and then 1m in the next dropdown

 

Frustrating.
A few things to try maybe.....
I take it you've ticked that pseudo mercator box in the editor, mine was unticked by default.
When I applied my lidar, I right clicked whilst in the tile to bring up the menu, then clicked lidar from there, may help?
I used Qgis 3.32.3, worked OK.
Are you seeing all the lidar entries in the menu in editor on the left when you import them?
I left the sea level box as 0 when creating my route in the UK, don't think this makes a difference.
With the reprojected tifs, do they appear back in the right geographical location when you place them back in QGIS? - I made a mistake with the coordinate system and they spawned near the equator.
For the coordinate system, I think in the end I used the one highlighted in the shot below (for East UK). However, I might have used the one just below it! I can't remember, sorry about that, but it was definitley one or the other.....was getting frustrated at that point.

The problem with this is that there's quite a few places to get slightly incorrect, as I found out - it's quite annoying.
So I used a combination of this tutorial, the video in this tutorial and some of DTG's video for the later stages.
Hope it helps.

 

thanks will take a look when I get home been dragged out to meadowhall

 

I was getting that solid block when the LIDAR data wasn't exactly how it needed to be. Make sure you follow the instructions in the first post to the letter, every single bit. If you find a different way to do something, don't! Follow it EXACTLY. And keep any eye on the bottom right in QGIS. It seems to like swapping the CRS data type without telling you.. or it did for me anyway.

 

what puzzles me most is why, for the editor, we have to use the .asc format when unreal engine uses .tif and .raw for importing height files. would've saved a lot of time and issues.

 

I think you're onto something here in the video tutorial i'm following he says to search for 28356 and the map in the corner of qgis highlights the place hes working on in australia so gonna look for the one that covers the uk

 

one thing i noticed in the tutorial i watched the guy only imported the .asc files but in Matt and Wills video he imported them into a separate folders from the .hgt files so i'm gonna do the same as Matt and Will see if it makes any difference I also chose, in qgis, to select a crs that covered the whole of the UK and not Australia , fingers crossed
EDIT: YES YES YES. thanks for your help Matt, Will, Fakenham, VictoryWorks and everybody here.

 

Great to hear you've got it sorted, it's cool when it works and shows all the cuttings and embankments. It's a long hard slog after this though, there's so much to learn. Just trying to find all the useful assets is a task in itself!

 

I was thinking about assets yesterday and wondering if you can create your own in blender or something similar to import or use ones from unreal engine marketplace. If possible i'm not sure how to use blender (so something i'm willing to sit and learn) and not sure how to get assets from epic games marketplace into our editor this i'm going to look up too. I don't want to be using just any assets, from what we have, I'm determined to make my route (although small) to be as close to the real thing as possible. One thing I have is determination. In 2001 I had a traumatic brain injury and doctors said after my operation I'd be in a wheelchair for up to 2 years maybe longer, I was so determined I was only in it for 6 month. Once I set my sights on something I have to get it done. EDIT: just another thought can we create assets in unreal engine 4.26 and then import them into our editor?

 

After several attempts I had success with the lidar. For me the issues was the flat raised areas mostly due to that I was importing lidar data too close to the edge of my source lidar data. So make sure you extend the source lidar a bit further out than the area being imported. Also with the CRS type as well you need to make sure you are setting that explicitly at every step that asks for this even if it shows the correct value at the lower right. Also before importing within the editor you need to set the "Use Pseudo Mercator" to "yes. When I missed the CRS was wrong the lidar worked but the terrain was all wrong.

But when I finally got it right the level of detail is worth the effort, and aligned very well with the real contours of the land.

 

UPDATED 10/10/23:: This is information for people creating a route that's in the UK. I followed along with (1) TSW4 PC Editor | How to import terrain - YouTube This video is a great start and to help others along. @ 4.00 to get UK lidar data go to https://environment.data.gov.uk/DefraDataDownload/?Mode=survey in the download options select LIDAR Composite DTM, Year 2022 or whichever the latest year is, resolution DTM 1M . @4.36 set the number to 27700 It should be something like OSGB36/BRITISHNATIONALGRID EPSG:27700 @11.36 go to this tutorial posted by Matt https://forums.dovetailgames.com/threads/plugin-creation.74410/#post-727523 and create your plugin for your route and then follow this tutorial .@1839 and @1908 Tick the Use Pseudo Mercator Box in both snap to landscape and Tile placement tabs Lidar settings dropdown. Make sure you setup your data, route and folders exactly the same. This is what worked for me I hope it helps.

 

I think I took on a bigger challenge as far as assets go the area i created is built up with housing and industrial buildings. My other route I was going to do is more flat and less built up , Skegness to Grantham, so until I learn more I'm going to save my Consbourgh to Swinton Route and go to skeggy haha

 

i need help with mine, ive done everything but it doesnt go to lidar it just says converted.

 

I tried following DonnyDave's guide for UK routes, I've got as far as getting HGT and ASC files in and working with Google Maps Overlay but UE4 doesn't seem to understand AUX or PRG files, am I doing something wrong? Also, as soon as I put the LIDAR data in, the map rises super high and all flat with no option to undo it.

I think after spending days at attempting to do this, that I might just wave the flag and surrender until Matt's guide comes out.

 

i've looked around and the aux and prg files aren't recognised in our editor so i'm deleting ones i'm doing atm will edit this post if it still works EDIT: It's OK to delete the .aux and .prg files as these are not needed in the editor

 

Haha my Lidar was totally fine all the time. I just forgot to set the checkbox... Thank you for your friendly reminder. I finally managed to import Lidar into TSW Editor.

Good point, but i don´t know how to add all my layers at once to the batch process. I can only add one by one and it still takes so much time... Isn´t there any possibility to batch all your TIF files in a folder at once to ASC files in another folder?
I did some research on the internet, but was not able to find any useful code for the batch process :/
Do you have an excample code which i can adapt to my file path?

 

I have already surrendered. I dont want to have a plug in folder full of incorrect data.

 

you don't need to import AUX or prg files

 

when your lidar raises like a block and flat its because there is no lidar data for that area, or it could be you've got the lidar but not the right co ordinates In your route creation. Best way to get the correct location lon/lat is opening Google maps and right click on so.ething, in my case I chose skegness railway station as it was at the area I wanted to start my route.

 

When you open the translate tool, there is a "run as batch process" button at the bottom left of the tool. Click it, then for files, click autofill, then select open layers and select all the layers to convert. Autofill the rest of the required fields then run.

 

that's really helpful thank you

 

I did all of that, I used Southampton Central as my point of reference (50.907257629569436, -1.4133854210332524), all the HGT files are importing fine, I can see river beds, the hills and the flatlands of the nearby docks. I used SU 41SW from the Defra Survey Data Download - LIDAR Composite DTM 2022 DTM 1M and followed the guides above but still flat lands on importing. Maybe I did miss a step but not sure which one

 

I know, it can be a painful experience. Have you used the following (highlighted in blue) UK based coordinate system when following the tutorials?

 

When you have got your hgt files make sure when getting your lidar you cover an area just bigger than the area you selected for your .hgt. also when creating your new route in the lon/lat for Southampton Central station just enter 50.90724 and -1.41335 there's no need to enter all of the numbers you've posted above

 

not sure why but i'm getting an error saying unable to save package because the file is read only has anyone else had and solved this ?

 

hi how do i change trains and times on pc editor?

 

I do just want to point out guys, I don't want to be that guy who gets annoyed because of this, but just to try to keep them sort of clean, this is a thread on LIDAR data if you have any other questions find the appropriate thread or make a new one. This is just so that we can separate all the threads and not get mixed up

 

A good point. its hard enough for some of us to find the help we're searching for without having to scroll through unrelated topics. As a suggestion can we have any tutorials put into a section that no one can comment on so they're easy to find as and when we need them?

 

A great tip from OlaHaldor that saves a lot of time in QGis when you have a lot of files. Right, so instead of importing all my geotif files, and then merge, I just merge the files off the bat. No reason to spend time waiting for the import, then wait for the merge.

Just import from folder instead.


After you've "re tiled" the merged heightmap and stored to a separate folder, you can batch convert the exported geotiff tiles like this. Click the "Run as batch process" at the bottom of the Translate window. Then click the "Autofill" under "Input Layer". Load the files by "add all files from a directory".


You will need to make sure that the files are stored as .asc as well.
Click the button with the three dots to pick a folder and a name. In my example I named it "dem_"


Once you're happy with that, a new box will ask you what you'd like to do:
Select "Fill with parameter values", and it will autofill all the output fields with something like "dem_<numbers and stuff that makes sense in a sequence>.asc"


Once that's done, click Run, and watch the magic happen.

 

I had issues with spikes in the lidar terrain, along the lidar tile borders.

Picked up from Lucas on the stream yesterday about the "-9999" and that solved my issues.

Step 7: Add -9999 to No data value for output bands

Before:


After:



Regards,
JT

 

Just to let everyone know I'm in the process of typing up a route editor route creation guide. I'm gathering all my information from these forums, YouTube and other sources and compiling it all into easy to follow step by step instructions (with pics) and once checked will work out a way I can get it available for everyone to download and follow after I get the green light from DTG to do so. I've got tomorrow off work so with a bit of luck and plenty of coffee I should get it finished.

 

Hi All,

I have followed the Lidar/SRTM part of the styream from the other night. FOr some reason when I apply my liodar data to the dem it just make sthese really big boxes, any ideas? Any help would be greatly appreciated.

 

You get those when there is no lidar data available for that area for the editor to find. It maybe that its just not available or the area you've selected to download wasn't selected that far

 

I got similar boxshape in my early tries, I believe it was in the reprojection step that I did wrong.

During reprojecting setup I follow this and had success:
Selected the merged layer.
Force Source CRS according to the data source
Target CRS to the EPSG:3857 Psuedo Mercator one
Nodata value for output bands: -9999 (fixing spikes in Lidar tile borders)
Output file resolution in target georeferenced units: 1.0

Start with a few tiles to check and expand. I wasted a lot of time on doing huge areas before actually doing it correctly...

 

This was the issue, thanks for the help guys!

Although I've now lost my map file in the editor so I can't reload it - can see it in WIndows Explorer though?

 

Not able to load in "file -> recent levels" either?

Try dragging the file from windows explorer window to content browser folder for the route in editor.

Its a know bug that files are not loaded so hopefully will be sorted next editor update.

 

Hi All, thanks for all your help and responses today - Have managed to make some great progress. Any ideas how to texture the terrain?

Thanks again!!