| David H. | it’s pretty incredible how much time is spent in GC |
| Jeremy K. | David – memcaching recordings gave 28% speedup on the dash |
probably bigger in production |
|
I think GC between requests will be a pretty decent speedup too |
|
but it’s harder to tell without deploying it |
|
since the next request isn’t paying the penalty for all the trash the previous request generated |
|
| Jason F. | i’m seeing GC thrown around a lot here. What does it mean? |
| Jeremy K. | garbage collection |
everything we do generates garbage :) |
|
and Ruby has to clean it up |
|
| Jason F. | Gotchya |
37green |
|
| Jeremy K. | we generate SO much garbage that Ruby’s little mom-and-pop garbage company is strained to its limits |
For some pages, we’re spending half of the request just doing cleanup |
|
So decreasing the collections we have to do is a huge boost |
|
| Jason F. | No kidding. Glad we noticed that. |
From our internal Campfire chat room.
John Topley
on 19 Jun 08Out of interest, how much RAM are you guys allocating to memcached? I heard that Facebook uses 3 TB spread across 200 servers, but that’s probably increased by now!
Tyler
on 19 Jun 08GC is actually something folks at my company think about quite a bit. Both .Net and Java, languages which are supposed to do GC on their own and not require memory management, actually do require memory management.
If you, as the coder/developer/programmer know that an object is finis, then you should by all means destroy it so that the garbage collector doesn’t have to. Proactive memory management, even if it is just in some of your code, will result in lower resource requirements.
Wells
on 19 Jun 08Ermmm GC is a hugely common thing for development shops to think about.
bryanl
on 19 Jun 08@Tyler: Ruby doesn’t provide a method to do manual garbage collection. I know there is some work being down to optimize the process, but I don’t think we’ll get access to a method to manually do it any time soon.
ML
on 19 Jun 08Fyi: Garbage collection (computer science) @ Wikipedia.
Joshua Warchol
on 19 Jun 08@Tyler: GC.start does a manual garbage collection, doesn’t it?
Matthijs Langenberg
on 19 Jun 08If you are facing design constrains in a high language like Ruby, because the GC isn’t doing a good job, that is a bad thing.
Joshua Warchol
on 19 Jun 0837s, how are you kicking off GC between requests?
BlogReader
on 19 Jun 08Tyler If you, as the coder/developer/programmer know that an object is finis, then you should by all means destroy it
How do you do that?
Eric
on 19 Jun 08@Tyler: I can’t speak for .NET, but the comment about destroying an object so the garbage collector doesn’t have to is not exactly possible in Java. You can ask the GC to do it’s job, and you can tweak just about everything related to how it does it’s job, but you cannot actually destroy an object.
Tyler
on 19 Jun 08@BlogReader – This is more of a ObjC, .Net or Java thing. In all three languages, you can leave the GC up to the VM/CLR, but doing so manually has its advantages.
In ObjC, you ‘release’ or ‘autorelease’ objects to destroy them.
In Ruby, is there a way to dereference an object so that it more readily apparent it can be garbage-collected? One problem that comes up in other systems is when you have a loop of objects which reference each other, and so none of them get GC’d. Most GCs won’t destroy an object if some other object is referencing it. This is one means of ‘managing’ memory in GC’d systems. If there is a way to dereference an object in Ruby, even if it is around, it may be easier for the GC to do its work.
Evan
on 19 Jun 08@ Tyler: I’m not a Ruby programmer at all but a quick bit of research seems to indicate that Ruby has a mark-and-sweep garbage collector, not a reference counting one, so the circular reference problem shouldn’t show up.
Evan
on 19 Jun 08@ Tyler again:
Again, not a Ruby guy, but I’m guessing you can set references to “nil,” or something along those lines, to have their objects vanish into the ether. Or just otherwise let them fall out of scope. This is the way it works in Lua, for instance.
Tyler
on 19 Jun 08To correct: In ObjC, you choose if you are going to use the GC or not. ‘release’ and ‘autorelease’ are used when not using the GC.
Bob
on 19 Jun 08Is Jason a programmer/developer? I hope he isnt because, he doesnt know what GC means. If he is a programmer/developer, then wow.
dusoft
on 19 Jun 08So glad, PHP does everything for you. Why worry about garbage collection? It should be automatic, anyway.
charles
on 19 Jun 08Even though you can’t directly control GC, you can tune around it. For example, some people allocate a huge heap size in java thinking that will help. What that does is allow alot of garbage to pile up, then the GC gets it all at once. Pity the poor transaction that gets stuck behind that! Better to have a smaller heap and allow GC to happen many times along the way… it spreads the pain out so evenly you don’t notice it.
Tim Moore
on 20 Jun 08It’s overly simplistic to say that it’s always better to manually release memory when you know that you’re done with it. Proactive memory management does not always result in less resource usage, if you’re considering all of the resources involved. Freeing memory isn’t free!
Modern garbage collectors are very heavily optimized, and as others have noted, some systems give you quite a bit of control over how it runs. The best strategy is going to be highly application-dependent, but in many cases you can get better performance - or at least better tradeoffs - with automatic garbage collection than you can with manual memory management. To stretch a metaphor: you don’t drive to the dump every time you have a small piece of trash to discard.
The point of garbage collection is not that developers don’t need to think about memory management, but that there is a separation between code that deals with managing memory and code that deals with implementing other kinds of logic. That allows for better reuse of code in different contexts that may not benefit from the same memory management strategies.
sandofsky
on 20 Jun 08_why has a great article on Ruby’s garbage collector: http://whytheluckystiff.net/articles/theFullyUpturnedBin.html
You can control the garbage collector. It’s just kind of limited to GC.start, and you should understand what you’re doing before you call it.
I love Ruby, so I’ll readily admit its problems. At the top of the list is its garbage collector. It just, well, sucks. But Rubinius looks like it’ll fix it.
Also, regarding Object-C, I’ve heard people call its retain/release model garbage collector. That’s reference counting. You call retain if you need to keep an object, and release when you’re done with it. When an object gets a reference count of zero, another thread runs dealloc for you.
Leopard introduced the option of a real garbage collector, but there isn’t one on the iphone if you’re curious.
JF
on 20 Jun 08Is Jason a programmer/developer
No, I’m a designer which is why I asked about it. I hadn’t heard the other guys talk about it this much before so I was curious what it meant.
Ethan
on 20 Jun 08@dusoft
Regardless if your language of choice takes care of GC for you, it should be of concern of whats actually going on in your program. Sooner or later it will rape you when you least expect it.
matt
on 20 Jun 08I tend to believe that understanding something about internals makes one a better developer. It reminds me of one of the most mis-quoted comp sci quotes “We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil”
If you look at the performance of your app afterwards you might miss small optimizations that compound to a huge issue. String concatenation in python for example:
http://www.skymind.com/~ocrow/python_string/
A 30x improvement in performance. Imagine if you find a multitude of “simple things” you can do in your code. That would significantly improve its performance.
Aaron
on 20 Jun 08@dusoft @Ethan
Ethan – you are totally right. PHP is great and takes care of lots of stuff, but it’s still easy to throw memory errors and run into memory leaks if you don’t clean up after yourself.
Especially if you’re on a budget (e.g. shared hosting) sql query optimization, unset(), and conservation of variables can be essential.
Dmitry Chestnykh
on 20 Jun 08The best comment evar. LOL
web design company
on 20 Jun 08Ah, these PHP programmers.
This discussion is closed.