• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Rob Spoor
  • Devaka Cooray
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
  • Tim Holloway
Bartenders:
  • Jj Roberts
  • Al Hobbs
  • Piet Souris

Does Python execution being slower than Java, affect its execution time when processing Big Data

 
Monica Shiralkar
Ranch Hand
Posts: 2542
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Python is slower than Java when it comes to execution time. Python is often used for Big Data processing e.g Spark etc. Does this effect performance in Big Data (with respect to the time taken )?

Thanks
 
Roland Mueller
Ranch Hand
Posts: 59
Android Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The workhorse for big data applications using Python is according to my knowledge not the Python bytecode rather than binary native libraries included in Python modules. This way I assume that performance difference should not be so dramatic.
 
Stephan van Hulst
Saloon Keeper
Posts: 13366
295
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You should never assume that one language is faster or slower than another. Execution speed is rarely a property of the language, but rather of the platform it is running on.
 
Tim Moores
Saloon Keeper
Posts: 7162
165
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

You should never assume that one language is faster or slower than another. Execution speed is rarely a property of the language, but rather of the platform it is running on.


Quoted for emphasis. In addition to that, to the extent than some language is actually consistently slower than some other language on a given platform, that alone still doesn't make it a knock-out criterion against it. There are likely other, more important, aspects that make this or that language preferable for a given task.
 
Monica Shiralkar
Ranch Hand
Posts: 2542
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:You should never assume that one language is faster or slower than another. Execution speed is rarely a property of the language, but rather of the platform it is running on.



Ok.I had read among the advantages and disadvantages of python that one advantage of Python is that development time is generally lesser and one disadvantage that execution is slower.
 
Tim Moores
Saloon Keeper
Posts: 7162
165
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

development time is generally lesser


That, too, is something which you simply can not say in general. I suggest to disregard whichever source these came from.
 
Monica Shiralkar
Ranch Hand
Posts: 2542
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok.thanks
 
Tim Holloway
Saloon Keeper
Posts: 24496
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you count debugging and testing time, the actual differences between languages tends to be about equal, excepting maybe assembler. WHERE you spend your time is another matter. Java, for example, requires a lot more time to be spent up front, less afterwards. Interpreted languages are generally quick to code, but more likely to blow up later in the lifecycle.

Execution speed, however is another matter. Python has addressed this in various ways, including "pyc" code and running Python in a JVM, where the same optimizations and JIT rules apply (more or less) as they do for Java.

But, as others have noted, a lot of the grunt work is not being done in Python, it's being directed By Python. If the heavy-duty work is in canned services running on a database server or GPU, then the speed of the Python part can dwindle to insignificance.
 
Monica Shiralkar
Ranch Hand
Posts: 2542
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote: Java, for example, requires a lot more time to be spent up front, less afterwards. Interpreted languages are generally quick to code, but more likely to blow up later in the lifecycle.Execution speed, however is another matter.



I was thinking this as speed of execution ( the time program takes to execute ).  If not the speed of execution , what is it ?
 
Roland Mueller
Ranch Hand
Posts: 59
Android Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote: Java, for example, requires a lot more time to be spent up front, less afterwards. Interpreted languages are generally quick to code, but more likely to blow up later in the lifecycle.Execution speed, however is another matter.



My point in the answer above is that Python packages do not necessarily only Python code. Instead Python often is only a binding layer around binary code written e.g. in C/C++. This is especially true for big data handling.

Thus, using a Python module for big data means that one is NOT using pure interpreted code or only Python.

Let's examine the content of python3-numpy rpm in my Fedora box: you can see a lot of shared objects (DLLs in Windows) included.

 
Tim Holloway
Saloon Keeper
Posts: 24496
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:
But, as others have noted, a lot of the grunt work is not being done in Python, it's being directed By Python. If the heavy-duty work is in canned services running on a database server or GPU, then the speed of the Python part can dwindle to insignificance.



As illustrated. For those who don't know Unix, those ".so" files in Roland's example are "Shared Object" files, which are equivalent to DLLs in Windows. Almost invariably written in C/C++ or assembler. Although I suspect Rust is also beginning to creep in.

And at the binary level, it's relatively easy to talk to the system's GPU(s), which is where the power to do things like bitcoin mining comes from. Oh, and fancy graphics, too.
 
Stephan van Hulst
Saloon Keeper
Posts: 13366
295
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I also don't think there's anything stopping anyone from writing a Python to assembly compiler.

I cringe a little when I read a Wikipedia page say something stupid like "interpreted language". Whether a language is interpreted or not is never a defining property of the language. All languages can be interpreted. All languages can be translated.
 
Tim Holloway
Saloon Keeper
Posts: 24496
167
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:I also don't think there's anything stopping anyone from writing a Python to assembly compiler.

I cringe a little when I read a Wikipedia page say something stupid like "interpreted language". Whether a language is interpreted or not is never a defining property of the language. All languages can be interpreted. All languages can be translated.



1. Easiest way would be to write a front-end parser for the gcc compiler just like Ada did. It would compile directly to machine code - compile-to-assembler is not very common anymore.

2. Actually, languages are generally defined and designed as either interpreted (e. g. BASIC) or compiled (COBOL). Yes, you can demonstrably flip the paradigm but you generally pay a penalty. I've seen BASIC compilers and interpreted Fortran, but the reason BASIC was invented was to provide a "Fortran" that ran in an interpreted environment. As a rule, interpreted languages tend to be loosely-typed and that means that a lot of things that can be nailed down in machine code at compile time in other languages have to be deferred and/or passed on to run-time libraries. Going the other way, interpreted Fortran is likely to be tokenized just like Java bytecodes.

Java is a good case in point, in fact. Originally designed in interpreter (token) form, the addition of Just-in-Time (JIT) compiling to machine code has been a major strength for it. Since then, others have jumped on that particular bandwagon (when they don't simply jump on the JVM itself).

Then there's FORTH. FORTH is a threaded code interpreter. You define stuff or enter commands, the commands are parsed and converted to thread pointers, the interpreter then steps though the pointers, calling each thread in turn via its pointer. Note that in this context, the threads are simply bits of executable code, not asynchronous threads (usually!)

So FORTH is a lot like Python invoking machine-language library functions except that the FORTH syntax is much simpler than Python's.
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic