• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

MySQL uses too little memory

 
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

I'm running through some bulk loading SQL procedures that are taking hours to run. It might be they need to take hours, but one thing I've noticed is that MySQL never requests much memory (~50 megs) throughout the process.

I know there's a dozen or so variables to increase buffer sizes, and I've tried a lot of them with little to no change. Any suggested settings I should look at?

-Scott
 
Ranch Hand
Posts: 2458
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
MySQL ships with several default configuration files. Start here: http://dev.mysql.com/doc/refman/5.1/en/option-files.html#option-files-preconfigured
 
Scott Selikoff
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Bauke, but I was looking for specific tips from people who have seen issues like this before. I've read through the MySQL documentation and its really not that helpful.
 
Bauke Scholtz
Ranch Hand
Posts: 2458
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So you have already tried using for example the my-huge.cnf instead? If that didnĀ“t utilize more memory, then the problem may lie somewhere else or there is actually not a problem at all.

At least, the variables are explained in detail here: http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html Most important are those variables with "buffer" in it. For example key_buffer_size and read_buffer_size.
 
Scott Selikoff
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Nope it does not. I'm bulk loading a couple gigs of data, tried using INSERT and LOAD DATA INFILE, but it seems to grind after a certain point. It is a single threaded execution though,

Again, I don't need a list of system variables, there's too many as it is and I've read through everything you've sent all ready. I'm just trying to find out if anyone has seen cases where mysql's memory size stays so small and specific ways to improve it.
 
Scott Selikoff
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Found the variable to increase memory size for INNODB systems: innodb_buffer_pool_size
It was set at ~10 megs... increasing it to 800 megs definitely increased memory size. Not sure yet about performance though.
 
Scott Selikoff
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Just as an update, changing the InnoDB definitely fixed the issue. As a benchmark, one bulk load went from 4 hours to 15 minutes. Definitely an vital setting to set if you want MySQL to really use the memory it has available.
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic