Tutorial Lidar Tutorial

Discussion in 'PC Editor Discussion' started by Lulon, Sep 29, 2023.

Tags:
  1. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    So, apparently importing LIDAR data is one of the hardest things with the TSW Editor.
    For the last days I have been trying everything to get it right.
    I watched the preview stream which only had very little information about LIDAR import.
    DTG Lukas gave some tips on the forum.
    But I think I finally got it.
    So I made a step by step tutorial :)

    This is just what works for me, I don't know if it works for everyone but at least this is giving a bit more info.
    And I guess this is probably not what would be the "official" way.

    If you follow this, I expect you to know a bit about the software QGIS (hopefully more than me),
    and have GDAL installed (don't know if it comes with QGIS by default).


    First off, you want your LIDAR data to extend slightly further than you actually need in the route.
    It should overlap a bit with the SRTM only areas.

    1. Get your LIDAR data in a format that QGIS understands.
    I used LIDAR data from NRW in Germany, which is in the .xyz format.
    There are other LIDAR formats which may require conversion first.
    Also note the Coordinate Reference System (CRS) that the data is in.
    Mine is in UTM zone 32N. The tiles are all 1km x 1km.

    2. According to DTG Lukas, import all you LIDAR data into QGIS.
    I imported all LIDAR tiles into QGIS (around 600).
    Depending on the amount this will take a while and you might need a good PC for this.
    Or maybe just to test this, choose a smaller area.

    3. After the import is done you need to merge all the layer together that were created on import.
    Go to Raster > Miscellaneous > Merge...
    Select all layers as input layers using the button on the right side.
    You might want to set it so that the merged file gets saved in case something goes wrong.
    Then click "Run"
    A new layer called "Merged" should have appeared.

    5. At this point you should quickly click on the little eye symbol and hide all layers to stop your PC from lagging.
    Collapse all layers and remove the imported layers. Only keep the merged layer.

    6. Right click on the layer in the list in the bottom left.
    Select "Layer CRS > Set Layer CRS..."
    In the new window search for the CRS your LIDAR data is in.
    Mine is ETRS89 / UTM zone 32N (EPSG:25832).
    Click OK.
    I honestly don't know if this step is neccessary as we set the source CRS in the next step anyway but just to be sure.
    If your layer disappears after this, right click on your layer and click "Zoom to Layer(s)"

    7. We now need to reproject the merged layer to the Pseudo Mercator system.
    Go to Raster > Projections > Warp (Reproject)...
    Select the merged layer as input layer.
    Set your source CRS the same as in step 6.
    Set the target CRS to "EPSG:3857 - WGS 84 / Pseudo-Mercator".
    I set resampling method to bilinear. Choose whatever you want.
    Set "Output file resolution in target georeferenced units" to 1.0.
    Just like in step 3 maybe you want to set it so that the reprojected file is saved somewhere.
    Click "Run".
    Now you should have the layer "Reprojected".
    You can try and add an OSM layer to see if it reprojected correctly.
    OpenStreetMap is in the browser above the layers.

    8. Now we need to tile the reprojected layer into smaller pieces.
    Open the processing toolbox by clicking Processing > Toolbox.
    Then on the right, open GDAL > Raster miscellaneous > Retile.
    As input layer, select the reprojected layer.
    Enter your tile with and hight. I used 3000x3000 but from a bit of testing, it doesn't seem to matter.
    But bigger tiles means less tiles but larger files.
    In one post DTG Lukas said he also uses 2500x2500.
    Choose your output directory.
    The output files will be in the GEOTiff format.

    9. The last step is the worst one.
    Delete the folder called "1".
    Delete every file that is only zero. (Thumbnail is black in Windows Explorer).
    Delete every file that is not 3000x3000 or whatever tile size you entered.
    I also deleted every tile that is more that half zeros.
    Convert every .tif tile to .asc using GDAL translate in QGIS.
    Yes, this is almost impossible by hand if you have many tiles, so you could write a script or
    search google for QGIS GDAL batch conversion.
    You can delete every .aux.xml and every .prj file if you want, as the TSW Editor won't accept those anyway.

    10. Check if the .asc files look good.
    Rows and cols should be 3000,
    cellsize should be 1.0
    upload_2023-9-29_3-8-0.png

    11. Open the editor and import the files in some empty folder, maybe next to your SRTM data folder.

    12. Now it's time to use the data.
    Go into Route Building Mode.
    Click on Snap Landscape To Track.
    I don't know what the "UTMZone" and "Use Pseudo Mercator" options do,
    but for me it's "UTMZone" = 32 and "Use Pseudo Mercator" = yes (was the default).
    Select all imported tiles in the editor.
    Click on "Use Selected Assets".

    Now don't right click the LandScapeStreamingProxys in the World Outliner.
    Instead right click on a tile in the main viewport.
    It must be a tile that you know has Lidar coverage in your data or else it will be flattened.
    You might not be able to see that its highlighted with a orange outline,
    but it's still selected and the context menu will pop up.
    Click on "Apply Lidar data to selected Landscape Proxys".
    The tile should now have LIDAR quality and you can clearly see the smooth transition from LIDAR to STRM.

    upload_2023-9-29_3-8-27.png

    The reason you have to select only one tile is because the editor will
    just flatten every selected tile isn't covered by LIDAR in your dataset.
    From then you just apply LIDAR tile by tile. Never apply to a tile out of your LIDAR coverage or it will be flattened.

    One more tip:
    In the top left of the viewport you can click on Perspective then on "Top".
    Then click "Lit" and then on "Visibility Collision" or something (Lit doesn't work in top ortho for some reason).
    Now you can navigate efficiently using the right mouse button while generating new tiles.
    You can also right click on tiles here and apply LIDAR here.
    upload_2023-9-29_3-8-52.png upload_2023-9-29_3-9-10.png

    upload_2023-9-29_3-9-32.png

    I hope this helps :)
    (and works)
     
    Last edited: Sep 29, 2023
    • Helpful Helpful x 14
    • Like Like x 12
  2. paintbrushguy

    paintbrushguy Well-Known Member

    Joined:
    Jun 9, 2020
    Messages:
    443
    Likes Received:
    672
    Godsend.
    I’ll have a crack in a moment (hopefully it doesn’t crash!)
     
    • Like Like x 1
  3. paintbrushguy

    paintbrushguy Well-Known Member

    Joined:
    Jun 9, 2020
    Messages:
    443
    Likes Received:
    672
    upload_2023-9-29_14-1-37.png
    You're a genius.
     
    • Helpful Helpful x 2
    • Like Like x 1
  4. KrisKol

    KrisKol Active Member

    Joined:
    Apr 2, 2023
    Messages:
    134
    Likes Received:
    72
    Lovely!!! :) YAY!! I am going to have another stab later on.
     
    • Like Like x 1
  5. protonmw

    protonmw Active Member

    Joined:
    Mar 18, 2022
    Messages:
    134
    Likes Received:
    91
    Lulon do you have also an instruction without SRTM data? I don´t want to register for data which I don´t need...
     
    • Like Like x 1
  6. KrisKol

    KrisKol Active Member

    Joined:
    Apr 2, 2023
    Messages:
    134
    Likes Received:
    72
    The lower Resolution SRTM creates Distant mountains, and other bits of distant lesser details. As far as I know that is.
    (its actually something that makes quite a bit of sense to me)
     
    • Like Like x 1
  7. JT-Nor

    JT-Nor New Member

    Joined:
    Sep 14, 2023
    Messages:
    17
    Likes Received:
    13
    Nice instructions, this will be my weekend project :D Thank you for sharing your findings!
     
  8. OldVern

    OldVern Well-Known Member

    Joined:
    Nov 2, 2020
    Messages:
    12,630
    Likes Received:
    25,346
    • Like Like x 2
  9. dtg_jan

    dtg_jan Community Manager Staff Member

    Joined:
    Apr 17, 2023
    Messages:
    488
    Likes Received:
    1,870
    Hello there,

    this looks incredible, well done! Pinned it is!

    All the best,
    Jan
     
    • Like Like x 3
    • Helpful Helpful x 3
  10. railroadamerica

    railroadamerica Active Member

    Joined:
    Nov 9, 2022
    Messages:
    78
    Likes Received:
    156
     
    • Helpful Helpful x 4
  11. OlaHaldor

    OlaHaldor Member

    Joined:
    Apr 10, 2019
    Messages:
    26
    Likes Received:
    35
    Excellent stuff. I've processed over 600 tiles to .asc this morning thanks to this, and it mostly works. There are some quirks with the LIDAR data which was surprising, but I guess that's the data, not the process.

    Thanks a lot for the guide :)
     
    • Like Like x 2
  12. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    I followed both the tutorial and the video, but there is something strange with my source data.
    The coordinates are not translated correctly so the asc files are filled with meaningless numbers.
     
  13. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Thanks for the positive responses! Im glad to see people having success with this.
    First part of the master class came early :D
     
    • Like Like x 1
  14. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Sorry, no. I tried it myself to use only LIDAR but the "Create Landscapes" button doesn't show up.
    But you only need to register for 30 m SRTM, not for 90 m SRTM as far as I know.
    And as KrisKol said, it's really useful.
     
    • Like Like x 1
  15. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    The only reasons why I think that could happen is that you didn't set the source and target CRS correctly during reprojection
    or your source LIDAR data is not in the CRS you think it is.
    Also I think it's really important to set output file resolution to 1.0 when reprojecting.
     
    • Like Like x 1
  16. OlaHaldor

    OlaHaldor Member

    Joined:
    Apr 10, 2019
    Messages:
    26
    Likes Received:
    35
    I figured out why I had issues. (sadly it seems Discord is down so I can't easily retrieve what I shared there in a group, but what it essentially boiled down to was corrupt/duplicate data within the same tile coordinate).

    When you convert TIF to ASC, there is a new folder created, labeled "1". As far as I gather, anything that goes into there is useless. Don't use it. Delete it once the conversion is done.
    I did not delete it, I imported the data to unreal, and.. That's when havoc started as I went on to apply LIDAR data on the tiles. It was really messed up in some places.

    I re converted but did not use the data in the folder "1", and now everything works perfectly! :)
     
    • Helpful Helpful x 3
  17. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Oh right, I forgot to mention that. GDAL retile always creates that folder and I think you can't disable it.
    I think the folder contains some zoomed out tiles if you want to use it in a google maps like system. Just delete it.
     
    • Helpful Helpful x 2
  18. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    Thank you. That was one of the issues and, based on OSM map, the coordinates are now correct.

    After the entire procedure, when I save the asc they still say NODATA_value.:|
    ncols 2500
    nrows 2500
    xllcorner 934221.330116000026
    yllcorner 5868595.380293999799
    cellsize 1.000000000000
    NODATA_value 9.9692099683868690468e+36
     
  19. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Okay, how weird. I never had that. And I can't think of a reason why thats happening. Maybe it has something to do with the data types? Leaving the data types as the default in QGIS worked for me. But I would check if the retiled tiles in .tif format are okay first before converting.
     
    • Like Like x 1
  20. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    If I check the properties, after the merge, there it is the strange number.
    Band1#long_name=GDAL Band Number 1
    Band1#_FillValue=9.96921e+36

    Never seen something so variable...
     
    • Like Like x 1
  21. protonmw

    protonmw Active Member

    Joined:
    Mar 18, 2022
    Messages:
    134
    Likes Received:
    91
    Even following text and video step by step, I can´t merge the lidar data. What´s wrong here? Source files come in EPSG:25833 *.xyz
    QGIS-Version: 3.32.2-Lima
    QGIS-Codeversion: c0b8833964
    Qt-Version: 5.15.3
    Python-Version: 3.9.5
    GDAL-Version: 3.7.1
    GEOS-Version: 3.12.0-CAPI-1.18.0
    PROJ-Version: Rel. 9.2.1, June 1st, 2023
    PDAL-Version: 2.5.5 (git-version: f1a9ac)
    Algorithmus gestartet um: 2023-09-29T14:54:05
    Algorithmus Verschmelzen startet…
    Input parameters:
    { 'DATA_TYPE' : 5, 'EXTRA' : '', 'INPUT' : ['M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5642_2_sn_xyz/dgm1_33376_5642_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5644_2_sn_xyz/dgm1_33376_5644_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5646_2_sn_xyz/dgm1_33376_5646_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5648_2_sn_xyz/dgm1_33376_5648_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5650_2_sn_xyz/dgm1_33376_5650_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5652_2_sn_xyz/dgm1_33376_5652_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5654_2_sn_xyz/dgm1_33376_5654_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5656_2_sn_xyz/dgm1_33376_5656_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33376_5658_2_sn_xyz/dgm1_33376_5658_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5638_2_sn_xyz/dgm1_33378_5638_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5640_2_sn_xyz/dgm1_33378_5640_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5642_2_sn_xyz/dgm1_33378_5642_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5644_2_sn_xyz/dgm1_33378_5644_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5646_2_sn_xyz/dgm1_33378_5646_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5648_2_sn_xyz/dgm1_33378_5648_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5650_2_sn_xyz/dgm1_33378_5650_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5652_2_sn_xyz/dgm1_33378_5652_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5654_2_sn_xyz/dgm1_33378_5654_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5656_2_sn_xyz/dgm1_33378_5656_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33378_5658_2_sn_xyz/dgm1_33378_5658_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5638_2_sn_xyz/dgm1_33380_5638_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5640_2_sn_xyz/dgm1_33380_5640_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5642_2_sn_xyz/dgm1_33380_5642_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5644_2_sn_xyz/dgm1_33380_5644_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5646_2_sn_xyz/dgm1_33380_5646_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5648_2_sn_xyz/dgm1_33380_5648_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5650_2_sn_xyz/dgm1_33380_5650_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5652_2_sn_xyz/dgm1_33380_5652_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5654_2_sn_xyz/dgm1_33380_5654_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5656_2_sn_xyz/dgm1_33380_5656_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33380_5658_2_sn_xyz/dgm1_33380_5658_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33382_5638_2_sn_xyz/dgm1_33382_5638_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33382_5640_2_sn_xyz/dgm1_33382_5640_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33382_5642_2_sn_xyz/dgm1_33382_5642_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33384_5638_2_sn_xyz/dgm1_33384_5638_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33384_5640_2_sn_xyz/dgm1_33384_5640_2_sn.xyz','M:/TSW/Freiberg_Nossen/DEM/dgm1_33384_5642_2_sn_xyz/dgm1_33384_5642_2_sn.xyz'], 'NODATA_INPUT' : None, 'NODATA_OUTPUT' : None, 'OPTIONS' : '', 'OUTPUT' : 'TEMPORARY_OUTPUT', 'PCT' : False, 'SEPARATE' : False }

    GDAL-Befehl:
    gdal_merge.bat -ot Float32 -of GTiff -o C:/Users/*/AppData/Local/Temp/processing_ddhmYt/774dd81a5a754687b6942eea55992655/OUTPUT.tif --optfile C:/Users/*/AppData/Local/Temp/processing_ddhmYt/120c0354ced84ca8afddc90627a2222b/mergeInputFiles.txt
    GDAL-Befehlsausgaben:
    Traceback (most recent call last):
    File "M:\Program Files\QGIS 3.32.2\apps\Python39\Scripts\gdal_merge.py", line 14, in <module>
    sys.exit(main(sys.argv))
    File "M:\Program Files\QGIS 3.32.2\apps\Python39\lib\site-packages\osgeo_utils\gdal_merge.py", line 632, in main
    return gdal_merge(argv)
    File "M:\Program Files\QGIS 3.32.2\apps\Python39\lib\site-packages\osgeo_utils\gdal_merge.py", line 554, in gdal_merge
    t_fh = driver.Create(out_file, xsize, ysize, bands, band_type, create_options)
    File "M:\Program Files\QGIS 3.32.2\apps\Python39\lib\site-packages\osgeo\gdal.py", line 2764, in Create
    return _gdal.Driver_Create(self, *args, **kwargs)
    RuntimeError: Attempt to create 10000x-17999 dataset is illegal,sizes must be larger than zero.
    Prozess lieferte Fehlercode 1
    Execution completed in 115.30 seconds (1 minute 55 seconds)
    Results:
    {'OUTPUT': 'C:/Users/*/AppData/Local/Temp/processing_ddhmYt/774dd81a5a754687b6942eea55992655/OUTPUT.tif'}

    Lade Ergebnis Layer
    Die folgenden Layer wurden nicht erzeugt.
    • C:/Users/*/AppData/Local/Temp/processing_ddhmYt/774dd81a5a754687b6942eea55992655/OUTPUT.tif
    Im 'Protokoll-Fenster' im QGIS-Hauptfenster sind mehr Informationen zur Ausführung des Algorithmus zu finden.
     
    Last edited: Sep 29, 2023
    • Like Like x 1
  22. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    The value "Band1#_FillValue=9.96921e+36" was the default to define the empty parts.
    But in the end the result is the same.
     
  23. fecrails

    fecrails Active Member

    Joined:
    Sep 23, 2023
    Messages:
    113
    Likes Received:
    145
    The detail level of LIDAR looks like it might be worth doing even in an area with completely flat terrain. The resolution seems fine enough to to pick up the contour of drainage ditches, road embankments, etc. If that's the case it would be a huge time saver in the long run even if a route has very little terrain variation.
     
  24. Lesethard

    Lesethard New Member

    Joined:
    Sep 21, 2023
    Messages:
    10
    Likes Received:
    1
    Thank you guys for your awesome work and tutorials.
    I´m sure there is still something wrong in my QGIS transformation process, because if i import lidar in asc format to TSW, nothing happens or the surface flattens.

    I think the problem is the merging process in QGIS. When in merge my Lidar data (1km x 1km - EPSG 25832 / ETRS89 UTM Zone 32N)
    in QGIS, the merged plate will be white, instead of highly detailed as shown in the tutorial video.
    Example:
    Picture 1 = Lidar Tiles (TIF-format) imported to QGIS
    Step1.png
    Picture 2 = Merged plate
    Step2.png
    Does anyone know what´s wrong here and how to fix it?
    Thanks in advance.
     
  25. Calidore266

    Calidore266 Well-Known Member

    Joined:
    Jun 17, 2021
    Messages:
    1,240
    Likes Received:
    1,767
    Wow! Very nice. It's cool to see examples of success.
     
    • Like Like x 1
  26. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    I don't know why that happens. I just tried it too with Munich data in GeoTiff format and for me it just works. Did you leave the data type as the default Float32?
     
  27. Lesethard

    Lesethard New Member

    Joined:
    Sep 21, 2023
    Messages:
    10
    Likes Received:
    1
    Yes i did. I also used the ETRS89 / UTM zone 32N EPSG: 25832 as Project CRS before merging.
     
  28. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Sorry I cant't think of anything else that causes it. QGIS complicated sometimes
     
    • Like Like x 1
  29. Lesethard

    Lesethard New Member

    Joined:
    Sep 21, 2023
    Messages:
    10
    Likes Received:
    1
    I just uninstalled QGIS 3.28.11 LTR - i thought the stable version would´ve been the right decision, but it seems it was not.
    Tried QGIS 3.32.3, and it seems to be much better now:
    tiles:
    upload_2023-9-30_14-14-34.png
    merged:
    upload_2023-9-30_14-14-17.png
    But sadly in TSW Editor, the tiles will be flat, when i try to apply lidar on SRTM. :(
    ASC Files are lookin good (no zeros):
    upload_2023-9-30_14-46-45.png
    Think i´ll wait for a official tutorial with an example from europe / germany.
     
  30. Boingleboeing

    Boingleboeing Member

    Joined:
    Jul 31, 2021
    Messages:
    85
    Likes Received:
    45
    Not sure but I think the minimum tile size is 2500 by 2500, have no idea if that is true but maybe try? Who knows it might work ;)
     
    • Like Like x 1
  31. Zeo2113

    Zeo2113 Member

    Joined:
    Dec 10, 2016
    Messages:
    52
    Likes Received:
    17
    Been trying to import Lidar data from Defra Survey (Gov UK site) for the Southampton to Portsmouth line, and I get all these wonderful file types and upon importing them into QGIS. I end up with nothing and no layers to merge. What files do these need to be in? Does each Lidar data site have different file formats?

    I see Matt is working on a tutorial to import Lidar and all of that so fingers crossed that may answer my question. The video did not help as much as I was hoping for, I only got as far as the 4 minute mark before things went strange.
     

    Attached Files:

    • Like Like x 1
  32. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Yeah I'm also excited for the official tutorial.

    For me the tiles are flat if the tiles you want to apply LIDAR to aren't actually covered by your imported LIDAR tiles. And make sure the reprojection is right. But looks like it from your Asc files
     
    Last edited: Sep 30, 2023
    • Like Like x 1
  33. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    The tif files should work
     
  34. Evolinox#6389

    Evolinox#6389 Member

    Joined:
    Aug 5, 2022
    Messages:
    29
    Likes Received:
    46
    Anyone else having Issues, when trying to Warp the merged Data? It always ends directly with error code 1?
     
    • Like Like x 1
  35. protonmw

    protonmw Active Member

    Joined:
    Mar 18, 2022
    Messages:
    134
    Likes Received:
    91
    Yes, see my post earlier in this thread. I can’t even get it merged.
     
    • Like Like x 1
  36. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    Just noticed that in the editor, under "Lidar Settings", "use pseudo Mercator" is an option, which is switched off in my case.
    So why it is critical if, as default, is turned off...
     
    Last edited: Sep 30, 2023
    • Like Like x 1
  37. joep#7206

    joep#7206 New Member

    Joined:
    Sep 27, 2023
    Messages:
    6
    Likes Received:
    1
    Is this code something to do with one or more variable sizes violate format constraints?

    If so I found that to fix it I needed to save the merged file to a file instead of a temp file.

    upload_2023-9-30_21-8-33.png
     
  38. jeffmorris

    jeffmorris Member

    Joined:
    Sep 26, 2023
    Messages:
    41
    Likes Received:
    14
    How do I get LIDAR data for USA? Which file format should I use? I found out that the STRM image was hidden behind the OSM image after unchecking the OSM image in QGIS.
     
  39. Daunfr59

    Daunfr59 Member

    Joined:
    Dec 8, 2016
    Messages:
    42
    Likes Received:
    41
    Thanks for the detailed tutorial which certainly helps with using QGIS.
    Sadly it hasn't worked for me as when I got to the last stage and selected my "home" tile the editor crashed with an error message EXCEPTION_ACCESS_VIOLATION reading address 0x0000000000000304.
    If I go through the process again the same thing happens.
    I downloaded 28 5km x5km tiles from DEFRA website (DTM 1 mtr) and merged them together.
    I then reprojected to 3857 and saved the merged landscape.
    I then retiled to 2500 x 2500 and ended up with about 280 tiles.(tif format)
    Deleted the half tiles from the edges and then batch converted to get 263 asc tiles.
    Loaded these into editor and clicked on use selected assets when they appear in the left hand box Ok.
    When I right click on one tile it says "applying LIDAR data" and then crashes.
    I will try checking the files again and reload into the editor as a couple of times it saved them and then saved them again .
    Thanks again for your tutorial / instructions - I think I understand what we're trying to do but not quite there yet !
     
    • Like Like x 1
  40. DonnyDave

    DonnyDave Active Member

    Joined:
    Jun 29, 2020
    Messages:
    220
    Likes Received:
    106
    everything worked for me till i created the new route all the data i got worked but when entering it into the editor mine too just showed a flat surface. Matt says he should have a tutorial setup by tuesday so gonna wait till then. I think my problem was i never had an option to choose EPSG:3857 - WGS 84 / Pseudo-Mercator
     
    • Like Like x 2
  41. Daunfr59

    Daunfr59 Member

    Joined:
    Dec 8, 2016
    Messages:
    42
    Likes Received:
    41
    Further to yesterday's post I didn't touch the LIDAR data but I did Uninstall and Reinstall the Editor to get a fresh start.
    I then set up a plug in for my route as Matt recommended and put the SMRT and LIDAR data into that. I created the low res tiles and this time it worked perfectly and the LIDAR data loaded - one tile at a time - exactly on the point chosen for the route origin. there are a few "knife Edge" ridges -probably along the edges of the tiles.

    I tried to "Save Current Level As" with a name other than Main but when I restarted the editor I can't get the Landscape back even though the tiles (LT x1-y1) etc are all still in the folder. Is there a correct way to save the Landscape or does it have to be remade every time ?
     
  42. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    I also have the knife edge ridges. I don't know why they're there. But you can remove them with the landscape tool.
    Regarding the saving, I just save using Ctrl+Shift+S. When I restart the editor the level file is not visible anymore but you can reopen it with File > Recent Levels > Your Level.
     
  43. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    Now I can say that the procedure with QGIS is perfect. Thank you OP.

    I tried in dozens of ways but nothing changed, until..... I found out my worst decision, ever.

    When an asc is ready can be imported into a "lidar" directory under the plugin.
    Until this evening I have always imported the file, added it "use selected asset" and tried to apply it to the tiles.
    The imported files have to be SAVED into UE4 format before using it, and the "pseudo Mercator" box needs to be TICKED and, finally....

    TSW4_00011.JPG

    On the right the HGT and on the left the infamous Lidar 1m :D.

    Next problem will be to delete the gigantic walls separating the Lidar data from the non Lidar.
    Unfortunately there are no Lidar data for 2/3 of my route :)
     
  44. GingerPitti

    GingerPitti Member

    Joined:
    Aug 17, 2022
    Messages:
    63
    Likes Received:
    67
    This worked pretty good. But why do my tile edges look like this?
    Screenshot 2023-10-03 024728.png
     
  45. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    Ahhh, glad you figured it out in the end :)
     
    • Like Like x 1
  46. Lulon

    Lulon Active Member

    Joined:
    Dec 3, 2017
    Messages:
    101
    Likes Received:
    116
    My edges also sometimes look like this. I don't know why. I wonder if when Matt posts his LIDAR tutorial, it would happen there too.
    But for now you can remove the edges using the landscape tool.
     
  47. Zeo2113

    Zeo2113 Member

    Joined:
    Dec 10, 2016
    Messages:
    52
    Likes Received:
    17
    I'm making some progress now on importing LIDAR data but it's very hit and miss. Anyone from the UK know which one I should be downloading?
     

    Attached Files:

  48. Daunfr59

    Daunfr59 Member

    Joined:
    Dec 8, 2016
    Messages:
    42
    Likes Received:
    41
    I first clicked on the National LIDAR Survey data Download which brings up a map of the UK. You can then home in on the area you want
    and there is rectangle selection button in the right hand margin .Select this and click on one corner of your area and it will then draw a red line and you can continue until you have drawn a rectangle around the area.
    Then click on "get available tiles" which will bring up a list of different surveys at the top.
    I selected Lidar Composite DTM -2022 _ Dtm1m (1 metre) and it will then list all the tiles in the area you have selected. I had 28 and opted to download one at a time.
    Hope this helps.
     
    • Like Like x 1
  49. bremen

    bremen Active Member

    Joined:
    Aug 24, 2023
    Messages:
    127
    Likes Received:
    103
    I have noted something that could be a bug or at least an issue (If somebody could replicate it).

    Terrain generated from HGT, then covered with Lidar data. Saved yesterday and opened this evening.
    I decided to extend some of the HGT terrain to have a better view and the usual procedure does not work.
    Top part of the image (lidar), then the original HGT terrain. The flat part is the "new" generation.
    Same HGT and I'm not outside the data contained in it.
    TSW4_00012.JPG

    Can be that the Lidar data needs to be applied only after setting completely the base terrain?
    If so, no extension can be made in a second moment...
     
  50. fakenham

    fakenham Well-Known Member

    Joined:
    Dec 9, 2016
    Messages:
    613
    Likes Received:
    885
    Looking to have a crack at this LIDAR business now I've had a play with track/materials etc. Has anyone had success with UK LIDAR data using these methods above? Or should I wait for Matt's tutorial - looks like he may be v busy though.
    Many thanks to Lulon the OP for posting this by the way.
     
    • Like Like x 1

Share This Page