stuart wrote in Wed Aug 22, 2012 8:55 am:Once I've got that done I'll take a look at this. As I said earlier, it may become less relevant if we improve the overall memory occupancy. The particular approach of generating a single huge OSG mesh for all the buildings may in fact be a dead-end and we may use something closer to the random trees - a relatively small number of buildings (say 60k) instantiated multiple times.
Thanks for the heads-up, I do understand how this is probably going to be less relevant then.
OTOH, exposing internal stats to the property tree would still seem useful, especially for people wanting to make bug reports, or even just for dynamic "feature-scaling" using Nasal and/or XML-Property Rules. Thus, I feel that even just replacing the SG_LOG() statements with SGPropertyNode->setStringValue(...) would be a good move, as it would help people looking "under the hood" of the system, so to speak.
And like I said earlier, being able to control the behavior of the system (even if that just means disabling/enabling allocations) would definitely be useful to see if segfaults/crashes reported by users can even be remotely related to the new system or not. If we had this option now, it would be much easier for us to provide support to the users here seeing "OpenGL out of memory" errors.
All that aside, your earlier comments regarding "swapping" all still hold true - Thorsten just confirmed that this is also what "kills" FG for him:
viewtopic.php?f=68&t=17114&p=164934#p164933In other words, I do think that it would be worthwhile to keep this in mind and possibly really come up with a subsystem that monitors FlightGear's RAM usage at configurable intervals of say 1-5 seconds, and dumps all the info to the property tree, where it can be further processed by Nasal scripts or GUI listeners. If it's really swapping that slows down or kills FG for so many people, we obviously need to prevent it - and having access to real time memory usage stats would definitely seem useful then - not just for the random buildings system, but also all other subsystems that may "over-allocate" and run into swap space.
Thus, even just having a boolean "/memory/swap/is-swapping" would be useful, because other subsystems could monitor it using a listener and then adjust their own allocation behavior dynamically.
EDIT: I can handle the Linux implementation of this ...
stuart wrote:The problem is that I'm not sure if we have access to the current real memory (as opposed to addressable space), because we'd want to stop generating buildings before we start swapping to disk.
Here's a first stab at a simple subsystem to monitor FlightGear memory usage on Linux at 5 second intervals, consider it a "proof of concept" prototype now, as this would need to be cleaned up and implemented for Mac/Windows respectively - on Linux it simply works such that it merely fopen()s /proc/pid/smaps and copies two metrics to the property tree:
- Code: Select all
diff --git a/src/Main/CMakeLists.txt b/src/Main/CMakeLists.txt
index 2a541e3..44953b1 100644
--- a/src/Main/CMakeLists.txt
+++ b/src/Main/CMakeLists.txt
@@ -20,9 +20,14 @@ set(SOURCES
main.cxx
options.cxx
util.cxx
+ ram_usage.cxx
${RESOURCE_FILE}
)
+IF(${CMAKE_SYSTEM_NAME} MATCHES "Linux")
+ list(APPEND SOURCES ram_usage_linux.cxx)
+ENDIF(${CMAKE_SYSTEM_NAME} MATCHES "Linux")
+
set(HEADERS
fg_commands.hxx
fg_init.hxx
@@ -34,8 +39,15 @@ set(HEADERS
main.hxx
options.hxx
util.hxx
+ ram_usage.hxx
)
+IF(${CMAKE_SYSTEM_NAME} MATCHES "Linux")
+ list(APPEND HEADERS ram_usage_linux.hxx)
+ENDIF(${CMAKE_SYSTEM_NAME} MATCHES "Linux")
+
+
+
get_property(FG_SOURCES GLOBAL PROPERTY FG_SOURCES)
get_property(FG_HEADERS GLOBAL PROPERTY FG_HEADERS)
diff --git a/src/Main/fg_init.cxx b/src/Main/fg_init.cxx
index 651b0d4..3c9d10f 100644
--- a/src/Main/fg_init.cxx
+++ b/src/Main/fg_init.cxx
@@ -116,7 +116,7 @@
#include "globals.hxx"
#include "logger.hxx"
#include "main.hxx"
-
+#include "ram_usage.hxx"
using std::string;
using namespace boost::algorithm;
@@ -1083,6 +1083,10 @@ bool fgInitSubsystems() {
////////////////////////////////////////////////////////////////////
globals->add_subsystem("properties", new FGProperties);
+ ////////////////////////////////////////////////////////////////////
+ // Add the ram usage statistics system
+ ////////////////////////////////////////////////////////////////////
+ globals->add_subsystem("memory-stats", new MemoryUsageStats, SGSubsystemMgr::INIT, 5.00);
////////////////////////////////////////////////////////////////////
// Add the performance monitoring system.
diff --git a/src/Main/ram_usage.cxx b/src/Main/ram_usage.cxx
new file mode 100644
index 0000000..1b1e567
--- /dev/null
+++ b/src/Main/ram_usage.cxx
@@ -0,0 +1,22 @@
+#include "ram_usage_linux.hxx"
+
+MemoryUsageStats::MemoryUsageStats() {
+ _mem = new LinuxMemoryInterface(); //FIXME: should be implemented for Win/Mac & Linux
+}
+
+MemoryUsageStats::~MemoryUsageStats() {
+ delete _mem;
+}
+
+void
+MemoryUsageStats::update(double dt) {
+ _mem->update();
+ double swap = _mem->getSwapSize();
+ double total = _mem->getTotalSize();
+ SG_LOG(SG_GENERAL, SG_DEBUG, "Updating Memory Stats:" << total << " kb");
+ fgSetInt("/memory-usage/swap-usage-kb", swap );
+ fgSetInt("/memory-usage/total-usage-kb", total );
+}
+
+
+
diff --git a/src/Main/ram_usage.hxx b/src/Main/ram_usage.hxx
new file mode 100644
index 0000000..e9febb9
--- /dev/null
+++ b/src/Main/ram_usage.hxx
@@ -0,0 +1,51 @@
+#ifndef __RAM_USAGE
+#define __RAM_USAGE
+
+#include <simgear/timing/timestamp.hxx>
+#include <simgear/structure/subsystem_mgr.hxx>
+
+#include <Main/globals.hxx>
+#include <Main/fg_props.hxx>
+
+#include <string>
+#include <map>
+
+using std::map;
+
+// Linux: /proc/pid/smaps
+// Windows: http://msdn.microsoft.com/en-us/library/windows/desktop/ms682050(v=vs.85).aspx
+
+class MemoryInterface {
+public:
+ MemoryInterface() {}
+ typedef map<const char*, double> RamMap;
+//protected:
+ virtual void update() = 0;
+
+ double getTotalSize() const {return _total_size;}
+ //virtual void setTotalSize(double t) {_total_size=t;}
+
+ double getSwapSize() const {return _swap_size;}
+ //virtual void setSwapSize(double s) {_swap_size=s;}
+protected:
+ RamMap _size;
+ std::string _path;
+ std::stringstream _pid;
+
+ double _total_size;
+ double _swap_size;
+};
+
+class MemoryUsageStats : public SGSubsystem
+{
+public:
+ MemoryUsageStats();
+ ~MemoryUsageStats();
+ virtual void update(double);
+protected:
+private:
+ MemoryInterface* _mem;
+};
+
+#endif
+
diff --git a/src/Main/ram_usage_linux.cxx b/src/Main/ram_usage_linux.cxx
new file mode 100644
index 0000000..673e564
--- /dev/null
+++ b/src/Main/ram_usage_linux.cxx
@@ -0,0 +1,49 @@
+// https://gist.github.com/896026/c346c7c8e4a9ab18577b4e6abfca37e358de83c1
+
+#include "ram_usage_linux.hxx"
+
+#include <cstring>
+#include <string>
+
+#include "Main/globals.hxx"
+
+using std::string;
+
+LinuxMemoryInterface::LinuxMemoryInterface() {
+ _pid << getpid();
+ _path = "/proc/"+ _pid.str() +"/smaps";
+}
+
+void
+LinuxMemoryInterface::OpenProcFile() {
+ file = fopen(_path.c_str(),"r" );
+ if (!file) {
+ throw("MemoryTracker:Cannot open /proc/pid/smaps");
+ }
+ SG_LOG(SG_GENERAL, SG_DEBUG, "Opened:"<< _path.c_str() );
+}
+
+LinuxMemoryInterface::~LinuxMemoryInterface() {
+ if (file) fclose(file);
+}
+
+void LinuxMemoryInterface::update() {
+ OpenProcFile();
+ if (!file) throw("MemoryTracker: ProcFile not open");
+
+ _total_size = 0;
+ _swap_size = 0;
+
+ char line[1024];
+ while (fgets(line, sizeof line, file))
+ {
+ char substr[32];
+ int n;
+ if (sscanf(line, "%31[^:]: %d", substr, &n) == 2) {
+ if (strcmp(substr, "Size") == 0) { _total_size += n; }
+ else if (strcmp(substr, "Swap") == 0) { _swap_size += n; }
+ }
+ }
+ fclose(file);
+}
+
diff --git a/src/Main/ram_usage_linux.hxx b/src/Main/ram_usage_linux.hxx
new file mode 100644
index 0000000..2fa2045
--- /dev/null
+++ b/src/Main/ram_usage_linux.hxx
@@ -0,0 +1,22 @@
+#ifndef __RAM_USAGE_LINUX
+#define __RAM_USAGE_LINUX
+
+ #include <sys/types.h>
+ #include <unistd.h>
+
+ #include "ram_usage.hxx"
+
+class LinuxMemoryInterface : public MemoryInterface {
+public:
+ LinuxMemoryInterface();
+~LinuxMemoryInterface();
+ virtual void update();
+private:
+ void OpenProcFile();
+ const char* filename;
+ FILE *file;
+};
+
+
+#endif
+