Python learnings

[Monica Shiralkar]

I have been programming in Java for several years. Recently since our project involved Python work I have been learning it.Below are some of my learnings :

Python is easy to get started with quickly.It is a scripting language, procedural as well as object oriented.It is interpreted language and not compiled one.

In Python programs, there is no mandatory entry point and it starts running from the top.Optionally we can provide a main function to be called when running as a script (instead of importing).

In python we don't need the formality of  writing var type for variables.For example we can write

output=process()

For which in java we would write var or int or String etc before the variable.

In python we don't need to put semicolon at the end of statement.

In python we don't need braces for functions, blocks etc .All we have to do is maintain an identation.

We can return more than one value from a method.

Any other essential things which I should learn next ?Thanks

[Monica Shiralkar]

I find it surprising that Python has same naming convention for function names as that of variables, that is words separated by underscores. Having different one would have made code more readable.

[Tim Holloway]

There are several different name-forming schemes, but most are based either on lower-case+underscore (C-like) or camelCase (Java-like), with some variations. Actually, I'm bad about mixing the two in Python. In general, when I'm using it in OOP form, I go for camelCase and when I'm doing procedural stuff, I go c_like, but don't expect consistency from me.

In Fortran, you didn't have case-sensitive names, Or underscores. I miss that sometimes.

[Jesse Silverman]

In Java, treating non-booleans as booleans is considered so bad that it won't even compile.

In Python, various objects have a "truthiness" about them that varies with their contents, and directly making use of this is a big part of normal idiomatic Python coding.

Some of them would be pretty surprising.

In general, because the list, tuple and dict are 'baked into' Python, and have language support rather than a library we all tend to have, there are explicit syntaxes for passing tuples, dicts, etc. in function signatures that are very different than Python...

Some of these lines should be edited, but here's what I did when I was playing "Truth or Dare" with the Python interpreter:

# How do empty lists, tuples and strings behave in if/elif/while? # Some empty and non-empty lists. great_singers = ['James Brown', 'Sam Cooke', 'Barry White'] great_autotune_singers = [] if great_singers:    print("Any non-empty list evaluates True") else:    print("So this will never happen.") if great_autotune_singers:    print("Empty lists never evaluate True") else:    print("Empty lists evaluate False.") empty_tuple = tuple() if empty_tuple:    print("Empty tuples, like empty lists never evaluate True") else:    print("Empty tuples evaluate False also.") str1 = "False" empty_str = '' if str1:    print("All non-empty strings, including 'False' evaluate true") else:    print("So this will never happen!") if empty_str:    print("All truly-empty strings, 'False' evaluate false") else:    print("This will always get hit, because the string is truly empty!") whitespace = '\t  \n' if whitespace:    print("Whitespace ain't 'empty', it evaluates 'True'") else:    print("So this will never run.") print('* ' * 6 ) false_str = 'False' b = bool( false_str ) if b:    print("All non-empty strings, including 'False' evaluate true")    print("Even on explicit conversion, 'False' and 'false' convert to 'True'.") else:    print("So this will never happen!")

[Jesse Silverman]

The calling conventions for functions are very flexible in Python and extremely different from the static OOPS languages (and C for that matter).

I still have culture-shock due to coming from a very uptight world where we were extremely fussy about knowing EXACTLY how we could be called, details of function overloading rules, etc.

Python has both positional parameters and named parameters which they insist on calling keyword arguments, also you can roll and unroll things into tuples or dicts right in the method definitions and calls.  This is all VERY different from Java, C/C++, C# etc.:

https://docs.python.org/3.9/tutorial/controlflow.html#more-on-defining-functions

Come to think of it, Java doesn't even support default parameters.  Python does but there are some gotchas that are specific to Python, I think also explained well on that page.

[Jesse Silverman]

Another thing you may "know" from Java where Python decided to behave differently.

In Python, division by zero results in an exception, ALWAYS, even when both are float type.

In Java, you can get +Infinity or -Infinity or NaN when doing division with floats or doubles or a mix of double or float and int, as per IEEE 754.

Not in Python.

The % operator also works differently than you are used to when the numbers are of different signs, giving different answers than Java does.
This is on purpose not by accident.

People sloppily describe "floor division" as dropping the fractional part, but it rounds towards -Infinity when you have different signs...

-5 // 2

>>> -5 // 2
-3
>>> 5 // -2
-3

So there are a few places where if you just picked up your Java code and dropped it into Python or vice versa, you would get some pretty unexpected behavior on some pretty basic operations.

I focused on these which I was just thinking about because for the most part knowing many languages helps you in all of them, except for the cases where one of them decides to have "better" or "more sensible" behavior.  The things I showed are definitely not errors, they were done on purpose, but are traps for people hopping back and forth from one language to another...

[Monica Shiralkar]

Tim Holloway wrote:There are several different name-forming schemes,

But isn't it good to follow the 1 naming scheme that is the one which
python website tells.


https://www.python.org/dev/peps/pep-0008/

[Jesse Silverman]

Hi Monica:

I'm working (yesterday and today) on getting Python's OOPS conventions to stop seeming weird to me.

As I think you spend more time in Java than anything else, this particular article might make sense to you:
https://realpython.com/oop-in-python-vs-java/

I have been reading other things first, however, none of them expected Java to be "what seemed normal", which that article does.

I feel people who did lots of C# will have a little less culture shock than Java-heads.
I look at the class definition and think "Where are the instance variables??  What are their types?"
It's all there in the conveniently named __init()__

Jesse

[Campbell Ritchie]

That looks a nice website, JS

I look at the class definition and think "Where are the instance variables??  What are their types?"

You cannot guarantee that the class' designer wants all the fields initialised from constructor arguments. Maybe some of the fields have sensible default initial values.

[Jesse Silverman]

I'm just learning here, but my initial take away (see what I did there?) was that those sensible defaults go into the constructor as well in Python.

What you don't see is the list of declared types and attribute names free-standing anywhere like C/C++/Java/C#

Still getting used to it, but that was my initial impression.

[Jesse Silverman]

I got my first correction in the Official Python tutorial today.

One about sequencing of presentation was closed due to "it was just me grousing" (it explains function-calling styles referencing dicts and tuples which haven't been introduced yet), two others are still open and haven't been closed or addressed by a change request, but hey, the official Python docs now are a teeny bit different because of my annoyingly detail-oriented nature.

https://bugs.python.org/issue43042

I wasn't joking, I was really staring at it thinking "Does it not do that for recursive calls also?"

I think the point here is that Python is definitely going to be a real third or fourth language for me in my life.

[Tim Holloway]

Campbell Ritchie wrote:That looks a nice website, JS

I look at the class definition and think "Where are the instance variables??  What are their types?"

You cannot guarantee that the class' designer wants all the fields initialised from constructor arguments. Maybe some of the fields have sensible default initial values.

The horrible thing about OO Python is that a Python class is a dictionary. Meaning that you can create instance variables within any member function without having named them explicitly in the class definition.

There is no concept of "public" and "private" members, so the distinction is made via naming conventions (starts with underscore).

And of course, it's one of those languages where "constants - aren't and variables - won't". No manifest constants.

Needless to say, these features mean that large-scale Python code can get quite fragile, which is why some of the major players have been forced to invent special tools to attempt to foil some of the issues that can arise.

[Jesse Silverman]

Tim Holloway wrote:

The horrible thing about OO Python is that a Python class is a dictionary. Meaning that you can create instance variables within any member function without having named them explicitly in the class definition.

There is no concept of "public" and "private" members, so the distinction is made via naming conventions (starts with underscore).

And of course, it's one of those languages where "constants - aren't and variables - won't". No manifest constants.

Needless to say, these features mean that large-scale Python code can get quite fragile, which is why some of the major players have been forced to invent special tools to attempt to foil some of the issues that can arise.

Isn't it much worse than you make it out to be, tho?
Not that I am grousing, I am trying to understand.

From what I understand, the default levels of protection despite _convention and __conventions, is that code OUTSIDE OF YOUR CLASS can create instance variables, you know, like, WHEREVER DUDE, and, for that matter delete them!  Maybe change a float to a list of strings for fun?

I have a good friend who is a good person, who somehow maintains the belief that in his favorite languages you can change the value of numeric (and other) constants is somehow a good thing, denoting 'flexibility'.  He finds Java, C++, C# etc. "stiff and uptight"...

I think it is the page I linked to, or something else from them one link away, that includes a reference to Guido Von Rossum saying "Hey, we are all consenting adults here."

Don't get me wrong, I love Python, but working on code that controls people's life savings, for instance, I feel like I might get slapped with a paternity suit if I'm not careful, despite being a consenting adult.

I am not sure how much the best linters protect you from these things, but my initial feeling is "kinda a lot", hence my interest therein.

[Monica Shiralkar]

Jesse Silverman wrote:

As I think you spend more time in Java than anything else, this particular article might make sense to you:
https://realpython.com/oop-in-python-vs-java

Thanks

[Monica Shiralkar]

Jesse Silverman wrote: but working on code that controls people's life savings, for instance, I feel like I might get slapped with a paternity suit if I'm not careful, despite being a consenting adult.
.

What does this mean ?

[Jesse Silverman]

Python lets you do a lot of stuff that would never ever compile in Java/C#/C++.
Some of it could easily do Very Bad Things that would have economic consequences, if you miss just a test or two.
That scares me a bit.

Of course, causing you to have children is NOT one of those likely consequences.
That specific consequence I feared was just a joke because the creator of Python dismissed these sorts of protections by stating we are "Consenting Adults".

[Monica Shiralkar]

Jesse Silverman wrote:Python lets you do a lot of stuff that would never ever compile in Java/C#/C++.
Some of it could easily do Very Bad Things that would have economic consequences, if you miss just a test or two.

Could not understand how. What kind of stuff ?

[Tim Holloway]

Imagine I'm a bank handling your recent use of a credit card while shopping. My logic subtracts $50,000 from your bank account to cover the binge on designer fashion accessories. Now imagine that someone crossed a character with a number and the program throws an exception before it can credit the upscale shop you visited. Now you are short $50,000 and so are they and you get harassing phone calls from them because you "didn't pay".

While this particular nightmare is usually covered by using database transactions, irrespective of the actual programming language, similar issues can arise that a strongly-typed programming language would have forestalled because that particular class of errors would keep the code from compiling in the first place.

[Jesse Silverman]

Exactly.  Even if the transactions don't go thru because we are saved at the database level or somewhere else, QA and users only see bugs that don't get caught before deployment.

Bugs showing up in the IDE are better than those caught and rejected by testing harness upon check-in, which in turn are better than those caught by QA, which in turn are better than those that make it into production and affect the business.

Unless one has perfect testing, which few do, there will always be bad things in weird corner cases that make it thru.

I prefer the languages I'm working in to move as high a percentage of anything I will ever do wrong to be just between me and a squiggle in an IDE, or at least a failed link (I remember when I was doing a lot of C++ that it was quite easy still to write some non-sense that would get caught out only at link time, not compile)...

Even in the little bit of mostly toy programming I've done to date in Python, I've made errors that showed only when the right line of code got hit with just the right data, that wouldn't have even compiled in other languages I've worked on a lot of Production code in.

Some of these (maybe a lot) could be prevented with the various Linters that are often in use for static type checking (among other things) combined with opt-in use of non-required Python features (which are mostly pretty new).

Python also allows you to do a number of things that I think sane developers writing code that needs to work should probably never do.
My sense is that various Linters will bark at you if you do these goofy things, they all seem to have a wide menu of opt-in complaints and let you make things informational messages, warnings or errors.

[Monica Shiralkar]

Tim Holloway wrote: similar issues can arise that a strongly-typed programming language would have forestalled because that particular class of errors would keep the code from compiling in the first place.

Since compiled languages have such advantage over interpreter languages what is the reason for languages like Python to be designed to not compile and instead run directly.Is it for rapid development.?

Python is an interpreted language and not compiled so how does IDEs like IntelliJ show compile time errors?

[Campbell Ritchie]

An interpreted language can not only be deployed quickly, but also needs less changing to run on different platforms.
IDEs are programmed to look for compile‑time errors; they show error messages for Java® code too, before you try to compile and run it.

[Tim Holloway]

Interpreted languages are often interactive languages. The lines do blur somewhat. You can be an interpreted batch language, like IBM's mainframe Job Control Language (JCL), you can run Java interactively (via groovy) and so forth. BASIC interpreters generally pre-compiled each statement as it was entered, but originally what they produced was bytecodes or similar tokenized language, not native machine code. Fortran is generally a compiled langiage, but there have been BASIC-like Fortran interpreters. .BASIC is sufficiently like Fortran in fact, that both were taught in the same University course I took.

When you compile a program, it's generally done in distinct stages. First comes lexical analysis. Then comes semantic analysis. Following this is contextual analysis. We'll ignore the stages after that, since they don't really matter here.

Lexical analysis (parsing) scans the input text and converts it to tokens. Java has a Scanner class that can do this, in fact. The lexical stage will generallty catch errors like naming a variable "00_LIKE", which is an invalid name in Python.

Semantic analysis analyzes the tokens in relationship to each other. For example, in Java, the construct "if ( a ! < b )" is lexically invalid.

Contextual analysis requires knowing about parts of the program that occur on more than one line. For example, if I invoke "fibonacci("X")" when "fibonacci" was defined as a function that takes an integer argument, then contextual analysis would reject that.

In the original time-shared BASIC software development systems, every time you typed in a line, the interpreter would do lexical and sematic analysis, but nothing more, You had to actually give the "RUN" command before anything further was done.

Python is generally even worse in that regard, because it's not commonly developed interactively, so no analysis at all takes place until you run the code. Even then, lacking strong typing indicators means that the "fibonacci" error I just mentioned wouldn't be detected unless you actually called "fibonacci". Java, on the other hand, would never have compiled it to begin with.

[Jesse Silverman]

Oracle hypes the two-phase compiled/interpreted thing that Java normally does (except on Android which compiles to DalVek and then to native code!) and that C# normally does (unless you compile it straight to machine code) as a feature, not a bug.

Originally, I was unimpressed by this.  Yes, you get portable bytecode out, but then you still have an interpreter/JIT compiler on the backend.

Why am I a convert?  During compilation, a tremendous number of real errors that real programmers actually make all the time are caught.  Your byte code is pretty much kosher at that time if your basic logic and formulas are correct.

The bytecode that comes out is good-to-go anywhere.
It can be interpreted / JIT-compiled-at-runtime-where-useful or translated right to a specific machine code after that, but you are already pretty sure you don't have basic language-usage errors right there while you are seeing your compiler output, in fact, in any modern IDE before that because both things that would fail a compile and other issues are checked even before that by plugins that understand the language and your environment.

The super-modern IDE's blur this.  Using a good linter on Python in Visual Studio Code, for instance, will catch a number of problems that wouldn't have been seen until you hit just the right line with just the right data.  They can be configured to emit a whole bunch of different informational, warning or error statements based on this.  They may do this when you choose to save (or auto-save before a run) or even while you are typing away in the editor. I am a big fan of this, because I don't want to check in any code that won't work right.

Historically, the features that made working on a large code base with many different developers working on the code relatively safe were perceived as "annoying" to people just trying to throw something together quickly to run a minute later.  Python was always ideal for that sort of quick stuff, but has been ever-increasingly used for larger projects.

Python has started adding a number of features that are aimed at ensuring code is correct in ways it didn't bother to look at before, mostly, but not all involving typing.  They are always opt-in because they don't want to break it as an ideal solution for quick-and-dirty stuff, which makes sense.  They are not interested in turning it into C++ or Java or C#.

So all the lines are now blurred compared to the 'good/bad old days" we are reminiscing about.

To answer your questions as to "If you guys are so upset about some of the differences Python and Java/C# why is it loved and used by so many people?"

In addition to being ideal for quick things you are throwing together to just run a few times yourself, Python was intended to have fewer weird behaviors and gotchas that would confuse those who are not developers for a living (but occasionally need to write some software that works) and would not be forgotten in between the gaps where people were not coding.  It has a very extensive standard library that helps you do a lot of different things easily, and an extremely extensive collection of modules written by others (sometimes in other languages) to enable you to just download and do even more.  A very large number of people are doing very advanced things with Python in data analysis and AI, some of which old guys wouldn't even consider programmers because there are all kinds of annoying things about programming that they don't know about and never have to think about....

A large part of preparing for what used to be the 815 exam are things that you don't even need to think about in Python.  A high percentage of everything you learn in Python goes directly towards "knowing how to do something useful and new" and less of what you NEED to learn in Python is about doing things safely and efficiently.

If one is trying to use Python the same way one uses Java, a higher percentage of one's time will be spent learning about features that affect correctness and efficiency.  That's where I am at.

It's not even strictly correct to say that Python is always interpreted, as there are things like PyPy that is actually Jit-compiling the same code, much like normal Java, but that is not 100% compatible with all the latest CPython code -- CPython is what you are using when you say you are doing "regular normal just plain Python" and that is wholly interpreted, with the confusing but nice aspect that Linters may be looking at your code for possible errors upon save before run or as you type in the editor.

[Rob Spoor]

Tim Holloway wrote:lower-case+underscore (C-like)

The official (?) term is snakecase. There are some more possibilties, see https://en.wikipedia.org/wiki/Naming_convention_(programming)#Multiple-word_identifiers. The ones I use most are:
* camelCase: Java, PHP, JavaScript/TypeScript
* PascalCase: C#, Go
* snake_case: Python, Ruby
* CONSTANT_CASE: constants in several languages
* kebab-case: Spring Boot properties

[Jesse Silverman]

Agreed with Rob except that snake case was very popular in C, and not unpopular in C++ before Python and Ruby were things.

Seeing all_the_snake_case in Python reminds me of doing C as a kid.

[Rob Spoor]

I never said that snake_case wasn't popular in C or C++, I just don't use those languages (unless when I have to do some JNI, but that's next to never).

[Monica Shiralkar]

Campbell Ritchie wrote:but also needs less changing to run on different platforms.

Like in Java we have to change forward slashes to backward slashes when switching between Windows and Linux ?

IDEs are programmed to look for compile‑time errors; they show error messages for Java® code too, before you try to compile and run it.

In case of langauge like Java, they do this using java compiler but in cases of interpreter , the compiler does not even exist so how do they do it ?

[Tim Holloway]

Monica Shiralkar wrote:
Like in Java we have to change forward slashes* to backward slashes when switching between Windows and Linux ?

No. Java has a universal filename path convention based on Unix. You can - and should - code Windows file paths using forward slashes in Java.

Like so:

String usersDir = "C:/users";
String fredsMusic = "fred/Music/Classical/Fuer Elise";
String fredsMusicFile = new File(usersDir, fredsMusic);

Not only does this aid in portability - the same code would transport directly to Linux, although you'd change "usersDir" to "/home". But it also avoids the treachery that typing the wrong number of backslashes in the file path can bring.
---
*REAL slashes!

[Monica Shiralkar]

In that case it is surprising that once I had written a code to read from file and when I took that code to linux I remember since it gave error I had to switch from forward slash to backward slash or the opposite.

[Monica Shiralkar]

Python generally requires writing lesser code than Java for e.g in Python we dont need to write variable type while assigning a value.

Despite this , in python writing like the first line below for a non string will give error and only the second will work.In java this is simpler.

print ("value of x" + x)
print ("value of x" + str(x)) 

[Jesse Silverman]

Unlike Java, Python will never let you store mutable objects in a set or as keys in a dict.

As we saw in other forums, if someone then changes those objects afterwards in Java, you can completely break the HashSet or HashMap the mutable objects are used in, losing them from retrieval or getting duplicate members or keys.

This is automatically prevented from ever happening in Python.

I think I mentioned this somewhere else, but apparently not in this thread.

[Jesse Silverman]

Every language I've done since C has had function/method overloading.

In normal Python, out of the box, there is no overloading of constructors or any other methods or functions.

I saw many presentations of introductory Python that never bother to mention this, the Python tutorial presumes you are probably coming from an already-established language like C or Pascal, not some brand new one like C++/Java.

There is incredible flexibility in how you can make calls to a function, but it is all handled in totally different ways than classical Java/C++/C# overloading.

Only when watching a presentation where they presumed most people probably knew Java did they come out and say this tho.

If you try to define overloads, only the last one defined will be seen by calling code.

Doh!

There may be some optional ways to do Java-style overloading, but they aren't standard or out-of-the-box.

[Jesse Silverman]

People either love or hate this guy.
Some can't understand how he talks, others think he talks too much and beats everything to death.
Lakhs of students just love him and say they never really learn anything until they get it from him...

He does say he will teach you Python even if you already know Java, because you will have many misconceptions about how OOPS works that don't carry over.
I thought it was just self-promotion, but started watching anyway and it is true.
https://www.youtube.com/playlist?list=PLd3UqWTnYXOkzPunQOObl4m_7i6aOIoQD

I had lots of misconceptions about how OOPS works that don't carry over well to Python, I have already had a few more cleared away, and had already lost a few before I got there.

[Jesse Silverman]

I said this before, but I am saying it again because it is even more true than I realized when I said it.

Basic entry-level beginning Python is pretty simple and basic still, fine.

If you want to know everything about Python, that is now a lot.

There is a lot of powerful and weird magic to modern Python 3.9.

I just started looking at Decorators and backed off except for their most simple and famous use cases.

Python is not the tiny super-simple language it was long ago.

I guess it has that in common with Java.

[Jesse Silverman]

Replying to my own earlier comment in this thread.

Don't know why I was surprised to see "no overloading" in standard Python.

1. Overloading resolution is a compile-time notion.  Python doesn't get compiled.
2. It is based on the signatures of the overloads, that is, on the statically declared types of each.
In Python, even mentioning the types of your parameters is a recent, opt-in-only thing.

So of course Python doesn't try to do overloading like C++/Java/Scala/C#...

Happy π day!

[Monica Shiralkar]

Jesse Silverman wrote:
Basic entry-level beginning Python is pretty simple and basic still, fine.

If you want to know everything about Python, that is now a lot.

Agree.

[Jesse Silverman]

Two weeks ago when Tim said:

The horrible thing about OO Python is that a Python class is a dictionary. Meaning that you can create instance variables within any member function without having named them explicitly in the class definition.

I smoothed over how wacky that is.

Two objects can be of the identical class, but since each *object* contains its own dictionary, you can't presume which attributes the object contains based on its class.
Hence the frequent use of the seemingly-unnecessary-but-not hasattr() built in function.

Still more delightfully, from the official docs:

(This is implemented by calling getattr(object, name) and seeing whether it raises an AttributeError or not.)

Well, isn't THAT special?

If you are writing all the code, you can make sure that you never do anything like that.

But it is perfectly legal Python.

One of the hard parts of learning Python "fully" for a mostly-static programmer is remembering all the things you will never do on purpose in Python, but that you or someone else could do at any time that would shatter your basic concepts of what you can assume you know about things.

[Jesse Silverman]

In the static vs. dynamic debate, most of the emphasis seems to be on correctness, and catching logical coding mistakes early rather than thru failing test (or production) results.

I think I said it somewhere before, but the dynamic stuff also greatly inhibits possibilities for optimization.
Some people keep thinking "If we compiled our Python, it could be really fast."

But because so many things we are used to knowing at compile-time really aren't known in Python until a line is executed, with particular data values, a lot of the benefits you'd get from compiling a static language just aren't there -- they of necessity can only be sussed out at runtime and couldn't be baked into generated code.

That feels and sounds like grousing, but it is kind of important in maintaining the appropriate mental models of what our programs and data objects even are when working in the different languages.