Tim Holloway

Saloon Keeper
+ Follow
since Jun 25, 2001
Tim likes ...
Android Eclipse IDE Tomcat Server Redhat Java Linux
Merit badge: grant badges
Biography
Long-time moderator for the Tomcat and JavaServer Faces forums. Designer and manager for the mousetech.com enterprise server farm, which runs VMs, a private cloud and a whole raft of Docker containers.
These days, doing a lot of IoT stuff with Arduinos and Raspberry Pi's.
For More
Jacksonville, Florida USA
Cows and Likes
Cows
Total received
196
In last 30 days
1
Total given
40
Likes
Total received
3135
Received in last 30 days
17
Total given
375
Given in last 30 days
8
Forums and Threads
Scavenger Hunt
expand Rancher Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Tim Holloway

The build instructions in the source archive are pretty detailed and they say you can build in 6GB of disk space, but it's disk-intensive so the faster the disk, the better, e.g., SSD.

For CPU/RAM:

Build instructions wrote:
At a minimum, a machine with 2-4 cores is advisable, as well as 2-4 GB of RAM. (The more cores to use, the more memory you need.) At least 6 GB of free disk space is required.



Have fun, and tell us how it went!
1 hour ago
I can't answer the first question, but I can offer some notes on the second.

As I've mentioned other places, I have heard rumors that very, very long ago there was a database developed under US Government contract. Back then, taxpayer dollars spent developing software, drugs and other Intellectual Property was public domain so that the people who paid for it could benefit. Assuming it wasn't classified information, anyway. This is also how Prime Computer was founded.

That, of course is no longer true. Everything has to be monetized and you pay once for the government work and again for the product.

Ranting aside, this legendary database became the foundation that Oracle was built on and so the story goes, PostgreSQL.

I cannot (haven't tried) to find the name or full story, but I have worked with a couple of similar databases back  in the early 1980s. One was written in FORTRAN, and when I tried to port it, it managed to crash the Fortran compiler, the linker, and the print spooler one after another when building. Probably why Honeywell gave up the minicomputer business early.

This ur-database predated SQL, I think, so there's been a lot of change since then, but yes, PostgreSQL is syntactically a lot like Oracle. They're definitely gone their own separate ways, and PostgreSQL, while still a general database has carved out quite a niche for itself in GIS systems.

In early days, MySQL was lightweight and convenient, while PostgreSQL strove for industrial capability. For example, I don't think MySQL supported transactions until about MySQL version 5. Nevertheless, MySQL was considered more than adequate for a whole raft of applications, and is probably the best-supported backend for many open-source projects to this day.

MySQL has the distinctive feature that it supports multiple database file formats for different purposes. All the way from simple CSV (!) files to InnoDB, which has historically been the most powerful backend. PostgreSQL has only supported one backend organization, and for years it frustrated me that its format would change so radically between even minor releases that you couldn't transfer the binary files from one server to another.

I don't have benchmarks, but the conventional wisdom is that MySQL is good for lightweight quick work and PostgreSQL is good for industrial-scale work¸ but there's a lot of overlap. Several systems I have have bounced back and forth between the two databases as new versions of the app were released. JPA applications are pretty much independent of what database you plug them into also.

I'm not sure if sharding is a feature of MySQL, but it can be used to assist performance by physically splitting the database based on frequency of access. Likewise, if I understand correctly, PostgreSQL has extra powers when maintaining a database across continents.

But by and large, unless you have very specific needs, you just pick whichever one you prefer. Avoid stored procedures and other vendor-specific quirks, and if you do have to switch, you'll be able to migrate relatively painlessly.
David Stephen,
Your post was moved to a new topic.
(This informational message will self destruct in two days)
MySQL is a proprietary product of Oracle Corporation. And presumably also a trademark. Ask Xerox® how that worked out.

MySQL was originally an independent company offering its product in both free open-source and paid support. Until Oracle bought their assets.

I'm not sure if Oracle has a free MySQL and less if it is open-source anymore, but FOSS OS distros are extremely reluctant to bundle in proprietary licensed code as part of their package archives. Note that you can't find Oracle's Java in those archives either, and instead have to download it straight from Oracle or use OpenJDK.

MariaDB was forked off from MySQL when Oracle ate MySQL and is completely FOSS. So far, the two products are essentially interchangeable. It remains to be seen how long that continues. Similar stuff happened when they bought OpenOffice and LibreOffice spun off.

Anil Philip wrote:

Mike Simmons wrote:Thanks!  I hadn't really considered it; in fact I just came up with most of it.  But perhaps it's worth doing something like that...


Honestly, I did not understand - it all went over my head. I am not very intelligent.
But I am convinced this is a shortcoming of Java.
I am thinking of playing with the Streams API, modifying the implementing code just to experiment and try out an idea.
Is it very hard to understand how-to, and modify and build the JDK on my local machine? Has anyone outside of Oracle done it?



Lots of people have done it, but remember that there's more than one JDK. Oracle has one, Open Source has at least one (OpenJDK), IBM has one, and so forth.

I wouldn't recommend building your own JDK just to understand streams. It wouldn't help much anyway because the gnarly thing about debugging streams is that the same code gets called over and over until your eyes glaze.

You can't easily obtain the source for Oracle or IBM J9, but anyone can "git clone" the OpenJDK source and build it. Like many open-source projects, it's just a matter of cloning or downloading/unzipping, running the "configure" to get it to detect your OS environment and build makefiles, then "make" to build it. Mostly it just takes a while to do all that compiling and such, since it's a large system.

2 hours ago
Maven really doesn't like it when you attempt to reference resources that are located in places specific to your local machine's filesystem and outside of the project directory. It breaks the "write-once/build-anywhere" paradigm that is central to Maven.

What I'd recommend is that you install your ojdbc4.jar into your local maven repository and reference it as an ordinary dependency. That's what I do.

Bill Pulling wrote:HI Joel.

I teach at a community college in London, Ontario, Canada, and for the past several years I have used MySQL as the back-end database for some JDBC work in my fourth semester Java course. I have never done it in the course yet, but I would like to incorporate picture storage as BLOBS into the database. Does your book deal with this area?

Bill Pulling,
Fanshawe College
London, ON



Bill, you might want to look at this example: https://gogs.mousetech.com/mtsinc7/gourmetj-springboot

It's a port to Spring Boot of an application that originally ran in Python on the desktop. The desktop version has been very useful to me over the years, but Python isn't as durable as Java when it comes to system components being upgraded and when Python2 went out, basically so did the app. I wrote a functional equivalent using JavaServer Faces that should age better, especially since I just brought the platform and libraries up to date.

The app keeps both large and thumbnail copies of uploaded recipe pictures in the database as BLOBs. It's using Spring JPA, so I hope that's not a problem. The current live demon at https://gourmetj.mousetech.com is running against a MariaDB database. The database code is essentially the same as it was when I used SQLite, which was the Python app's original database (Yay, JPA!)

If I don't have the MySQL schema file in there, I do have it scheduled to commit shortly.
There's a bit of a difference here. Java, in its "proper" forum is entirely defined within a single directory tree. However, OpenJDK and certain others do various tricks to make it appear as though the components are LSB-compliant. In my case, I define a JAVA_HOME, and put $JAVA_HOME/bin into my PATH and skip by the LSB, because I developed habits that predate that approach and besides, quite a few tools need a JAVA_HOME defined.

Python doesn't work that way. And incidentally, I think all my major machines, excepting the CentOS 7 ones run "/usr/bin/python" as a Python 3. Since Python has never as far as I know been anything but LSB-based, the libraries are separated by Python versions without regard to library versions. If you want a different default Python, use the alternatives system. That's what it's designed for.

I'm always a little leery of components that can be installed by either system package manager or product-specific installer.  Afraid I'll get them crossed. Although Python has enough internal versioning that I think it can deal with that.

Stephan van Hulst wrote:Doesn't Selenium just test what appears in the browser? Doesn't the application at least need to be deployed on a test server for that?



I believe so. Although I've been reading up on Spring Integration Testing and I think it can maybe cover some of that.

One problem is that a lot of modern web GUIs need both a remote JavaScript engine and at least some simulation of host logic to respond. Selenium itself can likely fake out the client-side stuff, since it's in JavaScript anyway, but you'd still need to emulate a server. Which, I suppose is possible. Ah, I really don't know what I'm talking about here. I'd have to go refresh my Selenium know-how. But it's possible.

Spring Boot can likely do more, since Spring Boot contains its own servet/JSP engine anyhow. A GUI like Swing, of course, has its own testing needs.

As a rule, I do most of my application logic in business beans, which can simply be unit-tested. When I really need to test a GUI, it's likely to be as part of testing an entire workflow.
3 days ago

Lou Hamers wrote:You can only have one 'python' at the system CLI level, but yes I don't think that's a big issue really, Java has the same issue. The nuisance I'm thinking of is all the 'pip install etc' commands that a lot of open source stuff tells you to run to get their thing working. I've had situations where I had a version of something installed needed for one thing, and needed another version of the same package for something other project.


That's not the case for either language. The easy way during Python2/Python3 days would be to invoke by name "python3". But the alternatives system, loathe it though I do, allows fine-tuning of that behavior. In both cases, a PATH override can select a particular version on a per-shell basis.

For libraries, it's a bit different. Unlike a Java Maven repo, installed Python libraries aren't versioned (which is curious, since Linux and Unix do support versioned OS libraries!) But there are mechanisms.

First and foremost, depending on how you pip-install, your selected library can go into the common system library, your own local account library or a local project. Most serious Python developers do, in fact, recommend setting up a Python virtualenv, and it shows how Python is a secondary language to me that I don't think I've ever actually done that. I don't have enough Python projects around to have had version conflicts and for that matter, have had relatively little breakage putting projects developed on the latest Fedora into CentOS 7.

Incidentally, when you put a wheel together to make an app pip-installable, it can define dependencies with versions. So there's that.

Bottom line, though, is that even though I'm no Python wizard, I really haven't had that much trouble with that sort of stuff. Which is good. Like Alan Kay said: "Simple things should be simple and complex things should be possible".

Lou Hamers wrote:
What I'm thinking of (something I haven't done at all yet) is actually coding/scripting the "pipeline" and shipping the data around from stage to stage. Other than the fact that Python has more libraries to call on, I'm not seeing how Java has a major disadvantage there (admittedly the former lack of native libraries disadvantage is a significant one).



My point of view is that I'd be more likely to just glue together microservices, in which case, what language or even what machine a given pipeline stage is in isn't really that important.

On thing for good or ill (and I've seen both), however, is that a scripting system can be edited in-place, whereas a compiled production system often entails more red tape and of course, building a deployable module. Since ML is more of a seat-of-the-pants sort of deal, and as mentioned, not everyone in it is going to be competent in IDEs, build tools and the like, it's hardly surprising that an interpreted language is preferable. While I sneer that the idea that programmers are going to be obsolete soon (I've been hearing that since the 1970s), there are some things that simply don't need the extensive software development expertise that something like an Amazon ordering system does.

Lou Hamers wrote:
I wonder how well a minimized/custom JRE (Java 9+) would do here. Given that they went to all that effort causing everyone big headaches during the process of creating JPMS, I would hope it makes Java competitive running in a low resource environment. At least I thought that was a main goal there!


We tried that with JavaME. It didn't go well. I know. I used to moderate the JavaME forum. Granted, Android's Dalvek is also stripped down, but my suspicion was that the biggest performance liability on the Pi was that its JVM didn't JIT as aggressively as the mainstream ports. As far as stripping it down, recall that the Pi was originally pitched as a "real" computer for the impoverished masses (been there/did that and I was pretty broke when I bought my first Pi, actually). A real computer deserves real software, and there are actually few apps in the Debian family that don't have a Pi package. And that's often as much due to failure to implement as it is lack of capability. It's ironic that I was able to emulate an IBM System/370 (probably more powerful than the original hardware), but there wasn't a 3270 terminal emulator app for the Pi. Not because it's complicated. Just because no one had bothered to port it.
Actually, TOAD is the weapon of choice we used for Oracle.

The only GUI tool I use for MySQL is SchemaSpy, which can turn any JDBC-compliant database (MySQL, Oracle, whatever) into a website map of the database schema. I do strictly CLI for MySQL administration. My production servers don't have windowing systems installed and I've never felt the need for a GUI desktop client.
I'll address this in reverse.

First, you can have multiple versions of Python installed in Linux, just as with Java. In fact, for a while, until Python 2 was killed off, having both Python 2 and Python 3 was a regular thing. It helped with the migration process by not dumping you from one to the other a là Visual Basic.

Secondly, Anaconda goes way back on Red Hat and Linux has a history of booting faster than Windows. Anaconda may actually be less bloated these days, since you no longer have to detect and configure as many different network interface, video, and disk controller types in this more standardized age. I'm not an anaconda expert, but I'm sure there are compelling reasons why such a critical process wasn't done in a compiled language. Red Hat has - at least before IBM ate them - never been part of the "Just Git 'er Dun!" software school and they've had plenty of time to convert had it been considered necessary. Of course, rebooting Linux is a lot less frequent than rebooting Windows…

The libraries I can pull in via pip have rarely given me problems, unlike CPAN. The pip utility is sort of like Maven that way.

And finally, as I said, you don't really "program" ML systems, you train them. The extra time and effort to edit/compile/debug versus simply edit-and-go is an unneccesary annoyance. The high-processor-usage is done in the "black box" and I neither know nor care whether that box is Python code, C code, or an FPGA co-processor.

I am a rabid fan of Java, or I wouldn't be such a perpetual nuisance here on the Ranch. But some things just aren't that great a fit for Java. I didn't do my recipe webapp in Python, but the IoT board I previously mentioned isn't powerful enough to run Java. It can, however, run CircuitPython and TensorFlow Lite.

Another hot platform for ML is the Raspberry Pi. I can attest that the Pi has been able to run Java historically, but the unit I had running my CNC machine had horrible performance using a Java GRBL interpreter and I was forced to replace it with a native-code app. JVMs are so hungry that the Amdahl/V6 mainframe I used to jockey back in the mid 1980's couldn't even begin to support a single JVM. So ML apps on the Pi are typically done in Python, because, again, no matter what the AI box is written in, the end users don't want to muck around with low-level code when they can do some simple Python instead. Since ML is less about programming than it is about training, a lot of people with no programming aptitude to speak of can get involved with it.




Joel Murach wrote:Howdy to all you code ranchers out there, ornery or not!

To answer Tim's question: With MySQL 8.0 and later, the utf8mb4 character set is the default, and it uses the utf8mb4_0900_ai_ci collation by default. So, the Swedish collation is no longer the default.



Thanks. I hope you've stressed that the "ci" part of that means case-insensitive. One of the things that blew up in my database migration was that I had ingredient types like "Baking Powder", "baking powder", and "Baking powder", and the lookup got annoyed at finding 3 matches where only one was expected. An unfortunate artefact of the original data. I had to remove the "ci" from the collation to make the port work.

The main worry about ML is that I can see a parallel with the infamous problem where people created advanced Excel spreadsheets containing small but significant errors and it became Garbage-In/Gospel-Out. I can't blame The Great Recession on that, since I was unwittingly part of the cause, and the fragile valuations we computed for Mortgage-Backed Securites were not done using any sort of AI or spreadsheet, but I've heard rumors enough on what a bad spreadsheet could do to a major corporation.

And the fatal flaw with ML is that it cannot extend beyond what it knows. I've failed at least one "programming aptitude test" because I knew more than the test creators and knew more than one "correct" answer while knowing the weaknesses of what they considered "correct".

The thing to realize about ML is that it's less about math (exepting perhaps statistics) and more about data. There are plenty of publicly-available ML engines out there. In fact, in front of my I have a PCB that has TensorFlow Lite built into it and it cost me about $16. Someday I plan to train it to send me an alert when its onboard microphones hear the "victory songs" of my laundry-room appliances.

What's really valuable and proprietary are the data sets. A good ML system needs LOTS and LOTS of data and the holders of that data would like to be paid for the effort of collecting it. Or, in cases like financial trading sets, want to have a competive secret for their machinations.
Very impressive! Have you considered packaging this as a testing library that could maybe go into the world Maven archives?
4 days ago