| Page 1 of 1 |
|
|
|
| Back to top |
|
 |
|
|
 Posted: Sat, 23rd Apr 2011 11:04 Post subject: |
|
|
Right, so, tried pooling the Strings as StringBuilders (you can't pool Strings), it failed miserably xD. Back to chunking the memory. God, this is so sad...
Sense Amid Madness, Wit Amidst Folly
|
|
| Back to top |
|
|
|
|
| Posted: Sat, 23rd Apr 2011 11:48 Post subject: |
|
|
Right, chunked everything, it's running fine now. Ran a test, memory plateaued at 150MB. Same test had previously taken 1.2 GB.
Moral of the story: There's a reason game programming is done in C++.
Sense Amid Madness, Wit Amidst Folly
|
|
| Back to top |
|
|
|
|
| Posted: Sat, 23rd Apr 2011 12:43 Post subject: |
|
|
Read through the links. Wow it sure is emberassing for MS as this is a problem quite some people could run into as it is fairly simply to recreate 
But I am sure that the .net team will fix this up. Such a ... oh wait, I checked the date of the article ... March 2009
|
|
| Back to top |
|
|
|
|
| Posted: Sat, 23rd Apr 2011 13:29 Post subject: |
|
|
Probably because constant allocation/deallocation of large chunks is a bad practice overall. You might want to consider allocating a large memory buffer/pool during the start of the program and manage memory yourself.
But if you split everything into equal sizes chunks memory will act as a pool anyway (i.e. wont fragment nearly as much). If you need extra performance, and want to completely avoid fragmentation, allocate just once and reuse.
|
|
| Back to top |
|
|
|
|
| Posted: Sat, 23rd Apr 2011 19:20 Post subject: |
|
|
Go away you bad person! (c++ programmer = evil)
|
|
| Back to top |
|
|
|
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Sat, 23rd Apr 2011 19:58 Post subject: |
|
|
| PumpAction wrote: | Read through the links. Wow it sure is emberassing for MS as this is a problem quite some people could run into as it is fairly simply to recreate
But I am sure that the .net team will fix this up. Such a ... oh wait, I checked the date of the article ... March 2009 |
And how do you suggest they deal with this problem exactly? This is the lesser of evils.
An interesting article:
http://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/ |
|
| Back to top |
|
|
|
|
| Posted: Sat, 23rd Apr 2011 20:11 Post subject: |
|
|
That's the article I was referencing to by saying march 2009 
|
|
| Back to top |
|
|
LeoNatan
Banned
Posts: 73193
Location: Ramat Gan, Israel 🇮🇱
|
| Posted: Sat, 23rd Apr 2011 20:22 Post subject: |
|
|
Well yes, but there really isn't a solution other than a one coming from the developer's mind.
Perhaps the "GC.CompactLargeObjectHeap()" offer that that derp is spammed in all articles and blogs on the topic (seriously, search for "GC.CompactLargeObjectHeap" and you'll see ), but that would lead to so much trouble, I am not sure it is worth it. Basically similar problems to HDD free space consolidation type of defragmentations.
|
|
| Back to top |
|
|
|
|
| Posted: Sat, 23rd Apr 2011 22:52 Post subject: |
|
|
| BearishSun wrote: | | Probably because constant allocation/deallocation of large chunks is a bad practice overall. You might want to consider allocating a large memory buffer/pool during the start of the program and manage memory yourself. |
Yeah but the entire point of C# is to prevent you from managing memory yourself. And for the record, I did try an object pool, it didn't work. I really don't know why. I ended up just chunking everything into objects that took up less than 85k.
Ran it overnight, is still working fine. Mem usage going up and down in between 100MB and 200MB. Cheers!
|
|
| Back to top |
|
|
[mrt]
[Admin] Code Monkey
Posts: 1338
|
|
| Back to top |
|
|
Werelds
Special Little Man
Posts: 15098
Location: 0100111001001100
|
| Posted: Sun, 24th Apr 2011 00:21 Post subject: |
|
|
GC's are good for apps that aren't particularly memory intensive, but even in large web apps you'll eventually have to do some of it yourself 
|
|
| Back to top |
|
|
|
|
| Posted: Sun, 24th Apr 2011 00:29 Post subject: |
|
|
|
|
|
| Back to top |
|
|
|
|
| Posted: Sun, 24th Apr 2011 04:21 Post subject: |
|
|
| [mrt] wrote: | What does granny say? If you want something done right, do it yourself? OoooOoo 
As much as garbage collectors are fun to be around with, they do have their flaws. Usual usage scenarios are handled with no major complaints, but as you saw in certain situations it can make the whole app (or even system) crumble down.
The only real solution is to manage memory yourself (as granny says) (and Bearish ), thats the only way you will ever get any satisfactory and expected and sought performance and reliability from your app. Know the tools you are wielding. |
You tha man! We handle a lot of C# scripting and relying on GC for anything is impossible. I'm not sure how it is for other apps but for what I've witnessed it's good just for handling small amounts of allocations (like mrt said, it works good for /most/ usage scenarios).
Even though we work with a lot C# we still use memory pools extensively. In our case it's because of GC spikes that cause huge FPS drops in our games. We don't do it because of OOM errors, but still it's a good practice, even with C#.
More "advanced" C# gets the slower it is (obviously, but not as little as you'd think). I worked for a company that always tried to be on top of technology, i.e. they ended up using LINQ as much as they could. And me (primarily employed as c++ programmer but ended up with C#) constantly complained about this and no one ever listened and then they ended up at a point where 200MB database was being processed for over 8hrs and you couldn't explain to them why it was so slow.
|
|
| Back to top |
|
|
|
|
| Posted: Sun, 24th Apr 2011 04:28 Post subject: |
|
|
| tainted4ever wrote: | | [mrt] wrote: | What does granny say? If you want something done right, do it yourself? OoooOoo
As much as garbage collectors are fun to be around with, they do have their flaws. Usual usage scenarios are handled with no major complaints, but as you saw in certain situations it can make the whole app (or even system) crumble down.
The only real solution is to manage memory yourself (as granny says) (and Bearish ), thats the only way you will ever get any satisfactory and expected and sought performance and reliability from your app. Know the tools you are wielding. |
What you say is right. It's just scary to think I've finished 3 years of Computer Science at a pretty good university, and memory management came up not once  Just goes to show how much a degree in CS is worth...
Oh, and an interesting point that comes up in relation to this issue: It doesn't happen in Java. So I guess .NET's usage of a Large Object Heap is detrimental in the long run? |
Depends on the GC I guess. Try using Mono  You'll cry.
Also, try reading Game Engine Architecture (by Gregory, not by Eberly), you'll learn why consoles perform so much better than PC with games (with the same hardware). There is no OS involvement, there is no virtual memory, it's all memory management algorithms at work. (Among other things ofc)
|
|
| Back to top |
|
|
| Page 1 of 1 |
All times are GMT + 1 Hour |
