• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

java.nio DirectoryStream vs java.io.File#list or File#listFiles performance and memory usages

 
Ranch Hand
Posts: 72
MySQL Database AngularJS Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I want to collect directory structure to a given depth (It can be huge recursive structure!).
Which approach should perform better and consumes less memory?

Approach 1: Using DirectoryStream




OR

Approach 2:



Q. As java doc says -


    * <p> Note that the {@link java.nio.file.Files} class defines the {@link
    * java.nio.file.Files#newDirectoryStream(Path) newDirectoryStream} method
    * to open a directory and iterate over the names of the files in the
    * directory. This may use less resources when working with very large
    * directories.



I am visiting directories in DFS manner(recursive approach), will java.io.File#listFiles perform better than DirectoryStream as stream entries won't be garbage collected till whole structure is visited (not sure on this) causing more memory usages?

 
Saloon Keeper
Posts: 15547
364
  • Likes 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Why are you worried about this? Do you have reason to believe whatever approach you're using is not sufficient?

Use the right tool for the right job, and then test if performance is acceptable, before wasting your time on trying to optimize parts that are fine as they are.

In this case, it looks like you want to walk a tree of paths. You even wrote a recursive function to do it. Instead, use Files.walk() or Files.walkFileTree().
 
shashank dwivedi
Ranch Hand
Posts: 72
MySQL Database AngularJS Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I wanted to list files at given depth only.
 
shashank dwivedi
Ranch Hand
Posts: 72
MySQL Database AngularJS Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am sorry. FileWalkTree already gives what I wanted. Just was a little bit concerned about memory usages as it also uses DirectoryStream.
 
Stephan van Hulst
Saloon Keeper
Posts: 15547
364
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
First test, then worry.
 
Sheriff
Posts: 22784
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

shashank dwivedi wrote:


That line alone should be enough reason not to use File.list or File.listFiles. Instead of throwing an exception, they return null. You have no idea why though. And it also doesn't return an empty array so you have to add the null check. When using Files.walk, Files.walkFileTree, Files.list or Files.newDirectoryStream, you get a proper exception that tells you why the files can't be listed / traversed.
 
I like you because you always keep good, crunchy cereal in your pantry. This tiny ad agrees:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic