SimulatorGuide < Netgroup

Simulator Handbook

Simulator Handbook

Intro

This is a discrete event driven simulator developed for P2P Streaming purpose but can be easly ported also for other P2P Service. This simulator aims to replicate the network status during a P2P streaming session of a large amount of users ( we hope > 10^5) for performance evaluation of the different kind of algorithm that exist in letterature or in common P2P streaming application (CoolStreaming, PPLive, Gridmedia ...)

Architecture

The architecture is composed of 3 layer:

User : it represents user behaviour (the time he connects and when he disconnects from the network)
Overlay : it represents P2P program behaviour (algorithm ...)
Network : it represents the net behaviour, with an optimized MMF approach (so for this moment we consider only TCP for chunck transmission).

Using the Simulator

The project is divided into 5 directory:

Engine: Contains the engine of the simulator, the Calendar, base events needed for the simulation and data structures for content distribution.
Network Contains the data structures for abstract network behaviour (methods for establish or destroy connections) and the algorithm for Max Min Fair bandwidth allocation
Overlay Contains the basic functions that every Node must do in every algorithm
User Contains some basic functions of the User layer that can be overloaded from user
Customize Contains all the classes that can change among different P2P alorithm so for evaluating performances of different application, you only need to change the files into this directory

There is also a simple grafical interface done with NCURSES lib. To compile the simulator you only need to run Makefile doing "make" or "make -f debug". You need pthreads and optionally ncurses libreries installed on your machine. To run the simulator, simply do "./simulator" and it will start the simulation with the parameters and functions specified into the Customize directory.

Programming the Simulator

Calendars and Events

As for the others discreet events simulator, the main component of its engine is a Calendar object that process other objects called Events according to the time they are associated. The routine for executing all events by the calendar is in main.cpp file and it's a simple loop that run all the instructions associated to every Event in the calendar, delete the Event and go to the next Event untill Calendar reaches simulation_length.

To put an event into the Calendar you must include the file "engine/Calendar.h" and do the static method:

Calendar::put(Event* e);

To get the more recent event from Calendar you must do:

Event* e = Calendar::get();

The calandar will go ahead and pass to the next Event. To view the current calendar time you can do:

Time t = Calendar::time;

Every Event must have a name, a time reference and a the set of instruction that must be executed in that time. In "engine/Event.h" you can find the prototype of the Event class that every event must be inheredit from all the other events. In "customize/Events.h" and "customize/Event.cpp" you can put your own events. In this example I write to the Trace file "Hello World!".

customize/Events.h

#include "../engine/Event.h"
class TestEvent : public Event {
 public:
   TestEvent(Time time, char* name);
   void body() ;
};

customize/Events.cpp

#include "Events.h"
#include "..engine/Tracer.h"
TestEvent::TestEvent(Time time, char* name) : Event(time, "TESTEVENT") {
   Tracer::write("Hello World!");
}

If you want to print the same string every 10 seconds, you use this trick: customize/Events.cpp

#include "Events.h"
#include "..engine/Tracer.h"
TestEvent::TestEvent(Time time, char* name) : Event(time, "TESTEVENT") {
   Tracer::write("Hello World!");
   Calendar::put(new TestEvent(Calendar::time()+10, "TESTEVENT"))
}

In "engine/Event.h" there are also some predefined basic events such:

EndDownload: It is called every time a node finish a download of a chunk
DisconnectNode: it programs the disconnection ad the distruction of a Node. This event is set
StartDownload: it programs the creation of a Node an its joining to the overlay network.
GenerateNodes: It is called every GENERATION_INTERARRIVAL_TIME and call User::generateNodes , a routine that generetes new nodes for growing the population until it reach a treshold NODE_NUMBER. It is a User mechanism and it will be explained better later.
PlayChunk: We expect that every node, after it connects to the overlay, start to download chunks from the network (Prebuffering). After he collects PREBUFFERING_THRESHOLD consecutive chunks, he start to play the stream. This operation is modelized as a PlayChunk events that every CHUNK_DURATION milliseconds check if the node has got a chunk. If he got it, next time it will be asked for chunk with successive chunkId, else LostStream function of the Node will be called and the node start PreBuffering.
StartDownload: It programs the starting of a chunk download.
CreateContent: It is called every CHUNK_DURATION milliseconds and add a ChunkInfo object (explained after) into a static list called Content and included in overlay/Chunk.h .

You can define Events that overload these basic events, for example to implements an algorithm every time a Node download a chunk, you can define a class like this:

customize/Events.h

#include "../engine/Event.h"
class ExtendedEndDownload {
    public: body() {
    EndDownload::body();  // Calls the "old" EndDownload body
    /* Your Alogorithm here
     *
     *
     */
     }
};

Output and TraceFile

As you see in previous example, if you want to write something in the tracelog, you must include "engine/Tracer.h" and write:

    Tracer::write("Test string");

A writer thread try to write the string on a file. The "log level" is setted as default to USER_DEFINED. If the LOGLEVEL (explained after) permits USER_DEFINED Logs to be written, the strings will be written in the tracefile.

The logging system allows flexible tracing, enabling/disabling logging for each category of events. These categories are listed in engine/Tracer.h. Special cateogries are also defined to allow skip tracing some events, or to trace events not included:

passing NO_LOG to Tracer's write function, i.e. calling Tracer::write(buf,NO_LOG), will not add buf to the log while the use of USER_DEFINED as a parameter will trace custom events. Tracer::write has USER_DEFINED has default parameter, so calling Tracer::write(buf) has the same effect of calling Tracer::write(buf,USER_DEFINED).

customize/Globals.h contains the variable to be set in order to specify the events to be logged. The variable's name is LOGLEVEL and must be set by summing the event's categories (also listed in customize/Globals.h) to be logged. Be careful to include LOG_USER_DEFINED category if user defined as well as default events need to be traced.

Example:

In order to trace "chunk creation" and "end download" events (as well as user defined events) LOGLEVEL must be set to "LOG_CHUNK_CREATION + LOG_END_DOWNLOAD " (see customize/Globals.h for log labels). New categories of events can also be defined. To do so add a new constant in engine/Tracer.h. Remember to add another constant in customize/Globals.h corresponding to 2^(new constant) in order to allow easy loglevel definition ( at the moment the Tracer can handle up to code 15). Once defined remember to pass the right constant when calling Tracer::write()

To set the name of the tracefile , the LOGLEVEL and other parameters, you must configure "customize/Globals.h" . The meaning of the parameters inside Globals.h file are explained inside the file.

User behaviour

According to the architecture of the simulator, In customize/User.h / cpp , there is the User class that represents the Users behaviour. This class have got two main methods

Time evaluateUptime()

and

void generateNodes()

. The former must give in its output the life duration of a node. This code, for example, make every node died one minute after their connection:

Time User::evaluateUptime() { 
   return Time(60); 
 }

You can use some of the statistical functions included in the Statistic class written in engine/Statistic.h / cpp files. The lattest methods are called every GENERATION_INTERARRIVAL_TIME and it must re-populate the node died during the simulation. For example if you want to mantain 1000 node that live of only 100 seconds, you can set GENERATION_INTERARRIVAL_TIME = 10 in Globals.h and write in User::generateNodes() this algorithm:

void User::generateNodes(){
        int diff = NODE_NUMBER - BaseNode::mNodes.size();
   Node* node;
   for (int i = 1; i <=diff; i++) {
      node = new Node(100, 100);
      Calendar::put(new DisconnectNode(node , Calendar::time() + evaluateUptime() ));   
   }
}

There is no reason for create more that an User. When a user is created it starts the first CreateContent and GenerateNodes events.

The overlay network

In overlay/chunk.h / cpp are defined three classes: ChunkInfo: A ChunkInfo object (and only one!) must be created for each different chunk passed to the overlay network. It collects some information such the Time it was created in the origin node or the chunkID. Every different ChunkInfo has different ID. Content: This class includes a static map of all the chunkInfo created during the simulation, and offers methods to find from a chunkId its corrispondent ChunkInfo object, to get the Last and the First ChunkInfo created. Chunk: Every node has a finite lenght list of downloaded chunks as a list of Chunk objects so there will be more Chunk object with the same ID. Every Chunk store some parameters needed for the Network part.

Every node, when they are created, call a connect method and before they are disconnected they call a disconnect method. You can customize the body of these methods (included in files "customize/Node.cpp") to evaluate your own streaming argument. These methos can call other methods, put some events in the Calendar, write something to the tracefile. For creating a Node, you can simple do Node(unsigned int a, unsigned int b) , where a is the upload bandwidth of the node (in kbps) and b is the download bandwidth (in kbps). In this example every node (except the Origin), starts to download 4 chunk from the origin node, in sequence.

void Node::connect(bool isMeTheOrigin){
   Tracer::write(std::string("NODE_CREATION ") + getNodeIdAsString(),NODE_CREATION);
   if (!isMeTheOrigin) {
      const int chunk_n = 4;
      std::map<unsigned int, ChunkInfo*>::iterator it = Content::mContentInfo.end();
      assert(Content::mContentInfo.size() > chunk_n ); // You haven't created enought chunk
      for (int i = 0; i <=chunk_n; i++)
         it--;
      for (int i = 1; i <=chunk_n; i++) {
         it++;
         Calendar::put(new StartDownload(Node::mNodes[ORIGIN],this, it->second ,(Calendar::time()+i)) );
      }
   }
}

If the node is created at the Time "30" and the last created ChunkInfo the 21, the node will start downloading chunk 17 at time 31, the chunk 18 at time 32,the chunk 19 at time 33 and the chunk 20 at time 33.

Several methods of Node are inheredit from BaseNode. These are:

getNodeId / getNodeIdAsString: return an the Node Id
getOrigin: return a pointer to the Origin node
getBwUp: return the upload bandwidth of the node (in kbps)
getBwDown: return the download bandwidth of the node (in kbps)
StartDownload: the node start to Download a chunk. It accepts as parameters a pointer to the uploader Node, a pointer to Chunkinfo, and optionally the first byte of the chunk and the last byte. These two last parameters allow to implements algorithms with chunks resume logics.

There is also a static map - called mNodes - that associates every node created during the simulation with their ID.

When you start a Download of a chunk, the network abstract layer will be responsable to calculate the end of this download (triggering EndDownload event) and to dynicamic recalculate the end of the this download every time the network status will change. Obviously in case of Node distructions, its uplink and downlink connection will be removed.

The Network part

There is not a really need for the user of this simulator to deeply know how the network part works. It allows you to simulate with a Max Min Fair approach, a TCP connection for the downloading of a chunk. Network reconfiguration mechanisms start automatically every StartDownload function and every EndDownload events.

Log Model

OPSS produces plain text ascii log in a file whose name is specified in the TRACE_FILE variable. In order to write something on the trace file, you need to give the following command:

Tracer::write(std::string("YOUR STRING ") , TYPE_OF_TRACE);

where TYPE_OF_TRACE can be one of the following constant:

NODE_CREATION 
NODE_DESTRUCTION
CHUNK_CREATION
START_DOWNLOAD
END_DOWNLOAD
PLAY_CHUNK
SIMULATION_INFO
DEBUG
USER_DEFINED
SIGNALLING
MESHINFO

In the Global.h file, you can tells the simulator which categories of events to trace. For example if you write:

#define LOGLEVEL LOG_MESHINFO+LOG_END_DOWNLOAD

You will enable the log only of the MESHINFO and END_DOWNLOAD events.

Each line in the log file follows the following pattern:

TIME_OF_THE_EVENT NAME_OF_THE_EVENT ADDITIONAL_PARAMETERS ...

Actually we use the following conventions for trace events' additional parameters:

END_DOWNLOAD

TIME END_DOWNLOAD DOWNLOADER_ID DOWNLOADER_BANDWIDTH_PROFILE UPLOADER_ID UPLOADER_BANDWIDTH_PROFILE CHUNK_ID CHUNK_CREATION_TIME DOWNLOAD_START_TIME (other less important staff)

NODE_CREATION

TIME NODE_CREATION ID_NODE

NODE_DESTRUCTION

TIME NODE_DESTRUCTION ID_NODE

The tracer class is a static class, and the writing procedure is performed on a separate thread.

-- LorenzoBracciale - 12 Sep 2006