Backend Implementation

• Tools and Dependencies

  • Linux
  • C++
  • Cpp System Monitor (API)
  • Cpp Command Output (API)
  • CSV

• System Monitoring Implementation

  • Overview

    EcoMeter includes a key feature of system monitoring, which involves a comprehensive process of information collection, data extraction, and live data calculation. The system monitoring feature is implemented by extracting relevant information from system files, parsing and processing the raw data, and then dynamically displaying the results to users in an easy-to-understand format.

  • Collecting System Information

    • System and Process Information

      System and process information is mainly derived from /proc virtual filesystem, which provides a mechanism for the Linux kernel to expose information about the system and its processes to user-space applications [1]. Each file or directory within /proc represents a different aspect of the system or a running process, and can be leveraged to obtain information of the system required.

      System information that obtained from /proc include [2]:

      • Kernel Information - /proc/version
      • Total processes / Running processes - /proc/stat
      • Up time - /proc/uptime
      • Memory utilisation - /proc/meminfo
      • CPU utilisation - /proc/stat

      Information of process with [pid] that obtained from /procinclude [2]:

      • Process ID - /proc/[pid] where pid is in any directory having an integer for its name
      • CPU utilisation - /proc/[pid]/stat
      • Command - /proc/[pid]/cmdline

    • Energy and Power Usage Information

      Energy usage data is extracted by utilising the Intel “Running Average Power Limit” (RAPL) technology, which is a feature available in modern Intel processors that allows the processor's power consumption to be monitored and controlled [3]. The /sys/devices/virtual/powercap/intel-rapl/intel-rapl:0/energy_uj file is part of the RAPL framework and reports the total energy consumption of the processor.

    • Other System Information

      • Operating System

        The operating system name is contained in /etc/os-release , which is a text file that contains information about the operating system distribution that is installed on a Linux system [4].

      • CPU Temperature

        The CPU temperature data is collected from /sys/class/thermal/thermal_zone0/temp , which is a system file that provides the current temperature reading of a thermal sensor on a Linux system [5].

  • Data Parsing and Extraction

    After learning about the above filesystems, we then use a set of utilities to extract and parse the raw information that we have collected.

    • The primary responsibility of the util class SystemParser is to parse the relevant information from the /proc filesystem. As mentioned earlier, the /proc filesystem contains a wealth of information about the running processes and the system itself, for example

      

      To extract only the necessary data from the lines of information, we utilise the SystemParser class, which allows us to filter the essential system data and present it to the users in a more meaningful format.

    • The class Command has a static method exec(), which is utilised to execute system commands and obtain their stdout results. The Command class works in conjunction with SystemParser to gather essential data from the system, particularly data that does not require parsing.

  • Live Data Calculation

    • CPU Utilisation

      With help of SystemParser , we can easily get the total amount of time that the CPU has spent in all states total_jiffies as well as the amount of time that the CPU has been active active_jiffies. From this information, we can, with a little effort, determine the current level of CPU utilisation, as a percent of time spent in any states other than idle by the following formula:

      $live\_utilisation = total\_jiffies\_difference\ /\ active\_jiffies\_difference$

      This works for updating real-time utilisation of every CPU core over a short period (one second in our case). To implement this, we continuously read the total_jiffies and active_jiffies of each core and calculate the utilisation over one second using the the following variant formula:

      $live\_utilisation = (curr\_active\_jiffies\ -\ prev\_active\_jiffies)\ /\ (curr\_total\_jiffies\ -\ prev\_active\_jiffies)$

      void Processor::UpdateUtilisations() {
          long curr_active_jiffies;
          long curr_total_jiffies;
          bool is_first_time = prev_active_jiffies.empty();
      
          for (int cid = -1 ; cid < LogicalCores() ; cid++) { 
              curr_active_jiffies = SystemParser::ActiveJiffiesC(cid);
              curr_total_jiffies = SystemParser::TotalJiffies(cid);
      
              if (!is_first_time) {
                  utilisations[cid + 1] = (float) (curr_active_jiffies - prev_active_jiffies[cid + 1]) /
                                                  (curr_total_jiffies - prev_total_jiffies[cid + 1]);
                  prev_active_jiffies[cid + 1] = curr_active_jiffies;
                  prev_total_jiffies[cid + 1] = curr_total_jiffies;
              } else {
                  utilisations[cid + 1] = 0;
                  prev_active_jiffies.push_back(curr_active_jiffies);
                  prev_total_jiffies.push_back(curr_total_jiffies);
              }
          }
      }
      

    • Process CPU Utilisation

      Similar to the method for calculating system CPU utilisation, we extract the amount of active times that CPU has spent on performing a process every one second, and calculate the real-time CPU utilisation of that process using the following formula:

      $cpu\_utilisation = (curr\_jiffies\_on\_process\ -\ prev\_jiffies\_on\_process)\ /\ (curr\_total\_jiffies\ -\ prev\_total\_jiffies)$

      void Process::SetCpuUtilisation(int pid, long curr_total_jiffies) {
          long curr_jiffies_on_process = SystemParser::ActiveJiffiesP(pid);
          if (prev_jiffies_on_process == 0 && prev_total_jiffies == 0) {
              cpu_utilisation = 0;
          } else {
              cpu_utilisation = (curr_jiffies_on_process - prev_jiffies_on_process + 0.0) /
                                (curr_total_jiffies - prev_total_jiffies);
          }
      
          prev_jiffies_on_process = curr_jiffies_on_process;
          prev_total_jiffies = curr_total_jiffies;
      }
      

    • Energy and Power Usage

      The data we collected from the RAPL framework is the total amount of energy consumption in microjoules (µJ) since the system is booted. By periodically sampling the energy consumption value and calculating the difference between consecutive readings, the power consumption of the processor over a given time interval (one second in our case) can be estimated using the formula:

      $power\_usage = energy\_usage\_difference\ /\ time\_difference\ (\times \ conversion\_amount)$

      To accurately determine the hourly energy consumption, our system takes into consideration various factors such as the possibility of system shutdowns and reboots, as well as the range of the energy counter. We achieve this by continuously updating the energy consumption data over a short period and monitoring any changes in the counter. Additionally, we have designed an algorithm that can detect when the energy counter has been reset to zero, keeping track of the number of times this has occurred. The hourly energy consumption is calculated using a formula that takes into account the current energy reading as well as any previous readings that were recorded in the current hour but during the last system boot, represented by the variable "extra".

      $curr\_hour\_energy\ =\ (energy\_counter\ +\ capped\_times\ \times\ cap\ -\ prev\_hours\_energy)\ (\times\ conversion\_amount)\ +\ extra$

      void Power::UpdatePowerAndEnergyUsage() {
          energy = EnergyUsageInUj();
      
          // Update power consumption
          if (energy < prev_energy) {
              capped_times += 1;
              curr_power_usage = (energy - prev_energy + MAX_ENERGY) * POWER_CNV_AMT;
          } else {
              curr_power_usage = (energy - prev_energy) * POWER_CNV_AMT;
          }
      
          // Update energy consumption
          prev_energy = energy;
          accum_energy_usage = energy + capped_times * MAX_ENERGY;
          curr_hour_energy_usage = (accum_energy_usage - prev_hours_energy) * ENERGY_CNV_AMT + extra;
      }
      

• Power Mode and CPU Scheduling Implementation

  • Overview

    Energy optimisation is implemented by allocating processes to appropriate CPU cores.

  • Identification

    To bind processes to CPU cores in order to optimise energy consumption, we begin by identifying the Performance cores (P-cores) and Efficiency cores (E-cores) on our Intel 12th Gen processor.

    const int Processor::PHYSICAL_CORES = std::stoi(raymii::Command::exec(PHYSICAL_CORES_CMD).output);
    const int Processor::LOGICAL_CORES = std::stoi(raymii::Command::exec(LOGICAL_CORES_CMD).output);
    const int Processor::HYPERTHREADED_CORES = (LOGICAL_CORES - PHYSICAL_CORES) * 2;
    const int Processor::E_CORES = LOGICAL_CORES - HYPERTHREADED_CORES;
    const int Processor::P_CORES = HYPERTHREADED_CORES / 2;
    

    We also identify the high-performance processes to be optimised by checking if its CPU utilisation is above or below certain bound.

  • Binding

    Given the processes that need to be optimised and CPU cores allocated, we use Command to execute taskset commands to implement CPU reassigning.

    void System::BindProcesses(vector<int> pids, int low, int high) {
        string taskset_cmd = "taskset -cp";
        for (auto &id: pids) {
            string full_cmd = taskset_cmd + " " + std::to_string(low) + "-" + \\
                                  std::to_string(high) + " " + std::to_string(id);
            raymii::Command::exec(full_cmd);
        }
    }
    

    BindProcesses(pids, 0, cpu.HyperThreadedCores() - 1); // Bind selected processes to P-cores
    BindProcesses(pids, 0, cpu.LogicalCores() - 1); // Bind selected processes to all cores
    BindProcesses(pids, cpu.HyperThreadedCores(), cpu.LogicalCores() - 1); // Bind selected processes to E-cores
    

  • Automate Optimisation

    The automation would be similar to a “black box” idea, in which the user is not directly involved but would receive updates on its functioning. This can be achieved through an algorithm that continuously searches for idle CPU cores, and identifies any suitable background tasks that can be assigned to them. The allocation of tasks to either P-cores or E-cores would depend on the specific requirements and usage patterns of the application. By efficiently utilising the available resources, this automated approach can optimise energy consumption and enhance system performance. This feature has not yet fully implemented, but we will keep working on it based on this proposed ideas in the near future.

• Data Storage Implementation

  • Overview

    A set of energy consumption data is stored in user’s local machine, which will be used to generate insightful report charts.

  • Log Files Path

    The log files would be created and stored in a user local directory where persistent application data can be stored. The specific location is determined by using Qt’s QStandardPaths::AppDataLocation .

    QString data_dir_path = QStandardPaths::writableLocation(QStandardPaths::AppDataLocation);
    

  • CSV File Structure

    • Hours log file (hours_usage.csv) contains two columns - hour and energy usage in Wh. It records the amount of energy consumption drawn in every hour in a day.

      hour,energyUsage
      0,0
      1,0
      2,0
      .
      .
      .
      22,0
      23,0
      

    • Days log file (days_usage.csv) contains two columns - date and energy usage in Wh. The date is formatted as YYYY/MM/DD and the file stores the total amount of energy usage drawn in a day.

      date,energyUsage
      2023/02/27,0
      2023/02/28,0
      2023/03/01,200
      .
      .
      2023/03/13,0
      2023/03/14,100
      

  • Data Access and Update

    The PowerDAO class provides a comprehensive set of methods for accessing and updating data that is stored in the log files, making it a vital component of the data storage layer. As the sole class responsible for interacting with the data storage layer, it serves as a protective shield, preventing other classes or components from accessing or modifying the log files directly, thereby ensuring data integrity and security.

Frontend Implementation

• Tools and Dependencies
  • C++
  • Qt
• Data Refreshment
• Live Data Display
• Energy Report Graphs Generation
• Daily Energy Cap Customisation
• Power Mode Selection

References:

[1] PROC(5) - linux manual page. [Online]. Available: https://man7.org/linux/man-pages/man5/proc.5.html. [Accessed: 17-Mar-2023].

[2] Itornaza, “CPP-system-monitor/readme.md at master · itornaza/CPP-system-monitor,” GitHub. [Online]. Available: https://github.com/itornaza/cpp-system-monitor/blob/master/README.md. [Accessed: 17-Mar-2023].

[3] “The linux kernel,” Power Capping Framework - The Linux Kernel documentation. [Online]. Available: https://www.kernel.org/doc/html/next/power/powercap/powercap.html. [Accessed: 21-Mar-2023].

[4] “os-release(5) — Linux manual page,” OS-release(5) - linux manual page. [Online]. Available: https://man7.org/linux/man-pages/man5/os-release.5.html. [Accessed: 21-Mar-2023].

[5] The Linux Kernel Archives. [Online]. Available: https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-class-thermal. [Accessed: 21-Mar-2023].