2024d_execution_model/sections/OurModel.tex

\section{Detailed Intermittent Execution Model}

In this Section, we provide a detailed description of our intermittent system execution model and its implications for software design.
In Sec.~\ref{sec:system_description}, we introduce the target architecture of our model and the reference system used for evaluations.
Sec.~\ref{sec:execution_model} presents our detailed execution model, designed based on the key observations from our experimental results.
In the following three sections, we discuss how this model affects both the power efficiency and correctness of software design.
Finally, in Sec.~\ref{sec:other_architectures}, we evaluate the effectiveness of our model across systems with different architectural configurations. 

\subsection{System Description}
\label{sec:system_description}

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figs/cropped/system.pdf}
    \caption{A typical hardware setup of intermittent systems.}
    \label{fig:hardware_setup}
\end{figure}

A typical intermittent system consists of two main components: a power management system and a computing system, as illustrated in Fig.~\ref{fig:hardware_setup}.
The power management system is responsible for accumulating the incoming energy into storage and providing a stable-voltage current to the computing system.
The computing system equips NVMs along with the MCU and peripherals, and utilize the NVMs for state retention between power failures.

This setup includes two notable decoupling capacitors that affect the execution model of intermittent systems.
The first one (C1 in the figure) is placed at the power management system as voltage regulators require a capacitor larger than the device-specific minimum capacitance for stable operation.
Also, the computing system has its own decoupling capacitor (C2) to stabilize operating voltage.

Recent studies increasingly explore 32-bit architectures for the computing system~\cite{shihIntermittent2024,wuIntOS2024,kimRapid2024,akhunovEnabling2023,kimLACT2024,kimLivenessAware2023,parkEnergyHarvestingAware2023,kortbeekWARio2022,khanDaCapo2023,barjamiIntermittent2024,songTaDA2024}, as emerging applications on intermittent systems, such as Deep Neural Networks (DNNs)~\cite{houTale2024,yenKeep2023,khanDaCapo2023,gobieskiIntelligence2019,islamEnabling2022,kangMore2022,leeNeuro2019,islamZygarde2020,custodeFastInf2024,barjamiIntermittent2024,songTaDA2024}, demand greater computational capabilities~\cite{bakarProtean2023a,carontiFinegrained2023}.
% Emerging intermittent applications demand increasing computing capability~\cite{bakarProtean2023a} due to computat
In this context, we employ a custom-built board featuring a 32-bit ARM Cortex-M33 processor (operating at 16Mhz) with 512KB of FRAM as a reference system.
For the power management system, we use a TI BQ25570 based system.
We empirically select XXuF and 220uF capacitors for C1 and C2, respectively, as these are the minimum capacitor sizes for stable checkpointing and recovery.
Sec.~\ref{sec:other_architectures} evaluates generality of our model in different architectures, such as systems equipped with MRAM and a 16-bit core.

% In this work, our goal is to model the buffering effects of these capacitors and evaluate their implications on software designs.
% (Recent studies present the need for better computing capability~\cite{bakarProtean2023a})
% For model validation and evaluation, we use a custom-built board equipped with an ARM Cortex-M33 core and 512KB of FRAM.
% Our setup requires XXuF and 220uF capacitors for C1 and C2, respectively, for stable execution of checkpoint and recovery.
% Sec.~\ref{sec:other_architectures} evaluates our model in different architectures.

\subsection{Execution Model}
\label{sec:execution_model}

\begin{figure}
    \centering
    \begin{subfigure}{\linewidth}
        \includegraphics[width=\textwidth]{figs/plot_expr_8a_cropped.pdf}
        \caption{Voltage traces for one power cycle.}
        \label{fig:execution_trace_one_cycle}
        \vspace{5pt}
    \end{subfigure}
    \begin{subfigure}{\linewidth}
        \includegraphics[width=\textwidth]{figs/plot_expr_8b_cropped.pdf}
        \caption{Voltage traces around the first power-on.}
        \label{fig:execution_trace_detailed}
    \end{subfigure}
    \caption{Voltages trace of energy storage and Vdd.}
    \label{fig:execution_trace}
\end{figure}

To derive general execution model with the effects of decoupling capacitors, we first present a sample measurement in our reference system.
To generate operation time of 50ms under 1.5mA current supply, we use a 470uF capacitor for the energy storage.
Fig.~\ref{fig:execution_trace_one_cycle} shows the voltage trace of the energy storage and the operating voltage (Vdd) of the computing system for one power cycle.
Note that Vdd is maintained by decoupling capacitors after current supply from the power management system stops.
The shaded areas represent the ranges that system executes the application code.
% Fig.~\ref{fig:execution_trace_one_cycle} shows the trace during one power cycle, and Fig.~\ref{fig:execution_trace_detailed} presents the first execution cycle in more detail.

Fig.~\ref{fig:execution_trace_detailed} presents the first execution cycle in detail. It shows several interesting differences between the traditional execution model and the actual operation.
Among them, we highlight three key observations that affect software designer's decision.

\begin{itemize}
    \item \textbf{O1}: The capacitor voltage drops quickly to charge decoupling capacitor when the system wakes-up ($t1$--$t2$).
    \item \textbf{O2}: The system executes at sub-voltage using the decoupling capacitors, even after power supply stops ($t4$--$t5$).
    \item \textbf{O3}: The decoupling capacitor discharges while the system is powered-off (after $t5$).
\end{itemize}


\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figs/cropped/detailed_execution_model.pdf}
    \caption{Detailed execution model of intermittent systems.}
    \label{fig:detailed_execution_model}
\end{figure}

% As we discuss in the following sections, all three observations significantly impact the performance of intermittent system designs.
% We propose a detailed execution model which reflects these observations.
Fig.~\ref{fig:detailed_execution_model} shows our detailed execution model, which reflects all the key observations.
When the capacitor voltage reaches the power-on threshold, the voltage experience quick drop due to the buffering effects, instead of gradual reduction.
After initialization, the system starts to execute at normal voltage (e.g., 3.3V).
When the voltage hits the power-off threshold, the power supply stops but system now starts to execute using the buffered energy.
Since voltage of capacitor decreases as it discharges, the system executes at sub-normal voltage until it reaches the voltage it cannot operate (e.g., 1.8V, typically known as Brown-Out Reset (BOR) voltage).
Finally, until the next power-on, the remaining energy in decoupling capacitors continues to discharge.

When designing intermittent systems with small capacitors, it is important for software designers to understand this model.
In the following sections, we discuss the impact of this model to software design in more detail.

\subsection{Impact on Power Efficiency}

The traditional model implies that power consumed between power-on and power-off thresholds are entirely used for the computing system.
However, our model reveals that considerable energy is used for charging the decoupling capacitors (\textbf{O1}) and dissipated during power-off durations (\textbf{O3}).
This implies that much smaller energy is used for the useful computation compared to the designer's expectation when using small capacitors.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figs/plot_expr_5_cropped.pdf}
    \caption{Distribution of energy consumed in a power cycle in different capacitor sizes (1mA current supply).}
    \label{fig:power_distribution}
\end{figure}

Fig.~\ref{fig:power_distribution} shows the distribution of the energy consumed for each stage of operation within one power cycle in various capacitor sizes, averaged over 50 executions. 
The line in the secondary axis represents the average operation times for application code.
The checkpoint is executed by the interrupt from the power management system, which is generated when the capacitor voltage reaches the power-off threshold (3.4V).
Note that this is the last point for checkpoint execution according to the traditional model.

The results shows that significant energy is wasted in decoupling capacitor.
The cost is more expensive when the capacitor size is small since the discharge rate follows the RC-discharging circuits.
While using smaller capacitors shortens the power-off durations, the discharging behavior penalizes them most since RC-discharging circuits discharge exponential (in our case, 50\% of energy is discharged at the first 161 ms).
As a result, 60.7\% of power is wasted in 470uF, and the rate decreases as the capacitor size increases, down to 28.5\% in 1320uF case.

More importantly, this wasted energy is expected to be used for the computation in traditional execution model, as all the energy except for the initialization and checkpoint/recovery is expected to be used in computations.
It brings significant errors between the two models in available energy for the execution.
In 470uF case, the actual energy efficiency (Execution) and the expectation from the traditional model (Execution and Discharged) differs by 4.99 times.

(Limitations of power failure injection and simulation based evaluations).

\subsection{Impact on Predicting Power Failures}

\begin{figure}
    \centering
    \begin{subfigure}{\linewidth}
        \includegraphics[width=\textwidth]{figs/plot_expr_6a_cropped.pdf}
        \caption{Input current is 1mA.}
        % \label{fig:eval_voltage_trace}
        \vspace{5pt}
    \end{subfigure}
    \begin{subfigure}{\linewidth}
        \includegraphics[width=\textwidth]{figs/plot_expr_6b_cropped.pdf}
        \caption{Input current is 3mA.}
        % \label{fig:eval_adaptivenss_finished_tasks}
    \end{subfigure}
    \caption{Ratio of sub-voltage operations in total execution time.}
    \label{fig:sub_voltage_execution}
\end{figure}

According to the traditional model, the system states should be saved to NVM before power-off threshold.
Our model shows that the system may operate after this point using the energy stored in the decoupling capacitors (\textbf{O2}). 
As a result, the energy storage voltage is not a good approximate of the remaining time that system can execute.

Modern MCUs can operate on wide range of operating voltages. 
Our case: 1.7V to 3.6V.

Fig.~\ref{fig:sub_voltage_execution} shows the ratio of sub-voltage executions over total execution times.
The x-axis shows the different capacitor sizes and the colors represent various power-off voltages.

\subsection{Impact of Sub-normal Voltage Execution}

The traditional model makes the software designers assume the system is executed under stable voltage.
However, the execution after the power-off threshold (\textbf{O3}) happens in sub-normal voltage.
Being aware of this is important to the software designers since the peripherals and analog components may function differently.

The two most critical examples are Analog-Digital Converter (ADC) and external memory.

\begin{figure}
    \centering
    \begin{subfigure}{0.45\linewidth}
        \includegraphics[width=\textwidth]{figs/plot_expr_2_cropped.pdf}
        \caption{Trace of one power cycle.}
        \label{fig:adc_error}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.52\linewidth}
        \includegraphics[width=\textwidth]{figs/plot_expr_3_cropped.pdf}
        \caption{Detailed trace.}
        \label{fig:fram_drror}
    \end{subfigure}
    \caption{Voltage of the capacitor and Vdd, sampled 470uF and 1.5mA.}
    \label{fig:adc_and_fram_error}
\end{figure}

Fig.~\ref{fig:adc_error}: ADC error.
Fig.~\ref{fig:fram_drror}: FRAM error.

\subsection{Sensitivity to Architectural Designs}
\label{sec:other_architectures}

Finally, we evaluate our model against two different architectural setups: MSP430 and Cortex-M33 with MRAM.
MSP430 has less computational capability than Cortex-M33 cores.
But it is a most popular platform for intermittent system researches, since it is a low-power system having on-chip FRAM.
We used MSP430FR5994 evaluation board, having 10uF of onboard decap.
For the second setup, we put MRAM to our evaluation platform instead of FRAM.
Core frequencies, capacitance of power management system, input power targeting about 50 ms execution.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figs/plot_expr_9_cropped.pdf}
    \caption{Energy breakdown and the ratio of sub-voltage operations in different architectures.}
    \label{fig:other_architectures}
\end{figure}

Fig.~\ref{fig:other_architectures} shows the results in different power-off voltage.
The bar in the left shows the energy breakdown in one power cycle, and the one in the right represents the ratio of the execution time operated in sub-voltage.