Win a copy of Design for the Mind this week in the Design forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

MySQL uses too little memory

 
Scott Selikoff
author
Saloon Keeper
Posts: 4010
18
Eclipse IDE Flex Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

I'm running through some bulk loading SQL procedures that are taking hours to run. It might be they need to take hours, but one thing I've noticed is that MySQL never requests much memory (~50 megs) throughout the process.

I know there's a dozen or so variables to increase buffer sizes, and I've tried a lot of them with little to no change. Any suggested settings I should look at?

-Scott
 
Bauke Scholtz
Ranch Hand
Posts: 2458
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
MySQL ships with several default configuration files. Start here: http://dev.mysql.com/doc/refman/5.1/en/option-files.html#option-files-preconfigured
 
Scott Selikoff
author
Saloon Keeper
Posts: 4010
18
Eclipse IDE Flex Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Bauke, but I was looking for specific tips from people who have seen issues like this before. I've read through the MySQL documentation and its really not that helpful.
 
Bauke Scholtz
Ranch Hand
Posts: 2458
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So you have already tried using for example the my-huge.cnf instead? If that didnĀ“t utilize more memory, then the problem may lie somewhere else or there is actually not a problem at all.

At least, the variables are explained in detail here: http://dev.mysql.com/doc/refman/5.1/en/server-system-variables.html Most important are those variables with "buffer" in it. For example key_buffer_size and read_buffer_size.
 
Scott Selikoff
author
Saloon Keeper
Posts: 4010
18
Eclipse IDE Flex Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Nope it does not. I'm bulk loading a couple gigs of data, tried using INSERT and LOAD DATA INFILE, but it seems to grind after a certain point. It is a single threaded execution though,

Again, I don't need a list of system variables, there's too many as it is and I've read through everything you've sent all ready. I'm just trying to find out if anyone has seen cases where mysql's memory size stays so small and specific ways to improve it.
 
Scott Selikoff
author
Saloon Keeper
Posts: 4010
18
Eclipse IDE Flex Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Found the variable to increase memory size for INNODB systems: innodb_buffer_pool_size
It was set at ~10 megs... increasing it to 800 megs definitely increased memory size. Not sure yet about performance though.
 
Scott Selikoff
author
Saloon Keeper
Posts: 4010
18
Eclipse IDE Flex Google Web Toolkit
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Just as an update, changing the InnoDB definitely fixed the issue. As a benchmark, one bulk load went from 4 hours to 15 minutes. Definitely an vital setting to set if you want MySQL to really use the memory it has available.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic