Update on local

IEEE-conference-template-062824.tex

@@ -11,6 +11,9 @@
 \usepackage{xcolor}
 
 \usepackage{subcaption}
+
+\newcommand*\circled[1]{\raisebox{.5pt}{\textcircled{\raisebox{-.9pt} {#1}}}}
+
 \begin{document}
 
 \title{Intermittent Systems at Small Scale: Execution Model and Design Guidelines \\
@@ -26,12 +29,13 @@ Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic
 \maketitle
 
 \begin{abstract}
-    Intermittent systems enable the execution of long-running tasks in environment with frequent power failures.
-    When designing such systems, software designers rely on execution models that abstract operations in hardware and describe how intermittent systems function.  
-    However, as recent techniques explore very short operation times with smaller energy storages, traditional models no longer provide precise abstractions of the real hardware behavior.
-    In this paper, we propose a detailed execution model that considers buffering effects of system's inherent energy storage components.
-    Our evaluation shows that intermittent systems designed without accounting for these effects can be up to 4.99x less power efficient and may lead to unsafe checkpoint execution.
-    Furthermore, our design guidelines improve end-to-end latency of applications in dynamic and static checkpoint techniques by 2.86x and 3.04x, respectively, without incurring any extra overhead. 
+    Intermittent systems execute long-running tasks in environments with frequent power failures, using small capacitors as energy storages.
+    Software designers rely on execution models that abstract hardware-level operations and describe how intermittent systems work.  
+    % When designing such systems, software designers rely on execution models that abstract operations at the hardware level and describe how intermittent systems work.  
+    However, as recent techniques target very short operation times with smaller energy storages, traditional models are failing to provide precise abstractions of the actual behavior.
+    In this paper, we propose a more accurate execution model that accounts for the buffering effects of a system's inherent capacitance, which is a major source of inconsistency in traditional models.
+    Our evaluation shows that systems designed upon the traditional model can be up to 5.62x less power efficient than expected and may lead to unsafe checkpoint execution.
+    Additionally, based on our model, we present design guidelines for small-scale intermittent systems, which improve the end-to-end latency of applications by 2.85x in dynamic and 3.04x in static checkpoint schemes, without any extra overhead. 
 \end{abstract}
 
 \begin{IEEEkeywords}

sections/Introduction.tex

@@ -2,19 +2,19 @@
 Batteryless systems are emerging as a promising future platform of Internet-of-Things (IoT) devices.
 These systems adopt a small capacitor as an energy storage and operate by harvesting power from environmental sources.
 This setup effectively addresses challenges associated with traditional battery-based systems, such as need for human intervention for recharging or replacement~\cite{choiCompilerDirected2022} and harmful environmental impacts~\cite{ahmedInternet2024}.
-These systems are also known as intermittent systems, since the computation happens intermittently only when there exist sufficient power to compute.
+These systems are also known as intermittent systems, since the computation happens intermittently during short time only when there exist sufficient power to compute.
 
 Intermittent systems require software support to sustain long-running executions across power failures.
 % An intermittent system requires software support to retain volatile system state information across power interruptions. 
 During operation, volatile data (e.g., registers or SRAM data) must be saved to Non-Volatile Memories (NVMs) through a process known as checkpointing.
-When power is restored, this saved state is restored to allow operations to resume from the context before the power failure (recovery). 
+When power is restored, this saved state is recovered to allow operations to continue the execution (recovery). 
 In designing these state retention techniques, software designers rely on an \emph{execution model} that abstracts hardware-level operations and represents behavior of intermittent systems necessary for software design.
 
-Figure 1 illustrates this execution model. 
+Figure 1 illustrates such execution model commonly adopted in literature~\cite{}. 
 As energy accumulates, the voltage of the capacitor gradually increases.
-And upon reaching the power-on threshold, the collected power is supplied to the system. 
-The system begins operation at this point, and execution is halted when the capacitor reaches the power-off threshold. 
-Software designers aim to leverage this execution model to implement intermittent systems at minimal cost.
+Upon reaching the power-on threshold, the collected power is supplied to the system. 
+The system begins operation at this point, and execution is halted when the capacitor voltage reaches the power-off threshold. 
+Software designers aim to leverage this execution model to implement intermittent systems at minimal cost (e.g., executing checkpoints just before power-off threshold~\cite{}).
 
 \begin{figure}[t]
     \centering
@@ -24,21 +24,23 @@ Software designers aim to leverage this execution model to implement intermitten
 \end{figure}
 
 
-In the meantime, researches on intermittent systems are increasingly exploring shorter operation time by using smaller capacitors (e.g., less than 1mF~\cite{ahmedEfficient2019}).
-Operate on small capacitors is generally desirable, as it not only reduces device volume but also enhances the responsiveness by enabling the system to wake up more frequently~\cite{bakarProtean2023a,maengAdaptive2020,alsubhiStash2024}.
+In the meantime, researches on intermittent systems are increasingly exploring shorter operation times by using smaller capacitors (e.g., less than 1mF~\cite{ahmedEfficient2019}).
+Operating on small capacitors is generally desirable, as it not only reduces device volume but also enhances the responsiveness by enabling the system to wake up more frequently~\cite{bakarProtean2023a,maengAdaptive2020,alsubhiStash2024}.
 As a result, recent studies have targeted operation times in the range of tens of milliseconds~\cite{reymondSCHEMATIC2024,wuIntOS2024,yildizEfficient2023,choiCompilerDirected2022} or even microseconds~\cite{reymondSCHEMATIC2024,wuIntOS2024}.
-However, as energy storage size decreases, the traditional execution model no longer provides an accurate abstraction of actual execution behavior.
+% However, as energy storage size decreases, the traditional execution model no longer provides an accurate abstraction of actual execution behavior.
+However, as energy storage size decreases, the traditional execution model is failing to provide an accurate abstraction of actual execution behavior.
 % The challenge is that the traditional execution model does not provide precise abstraction of the real execution anymore when the energy storage is very small.
-The major source of this discrepancy is the buffering effect of decoupling capacitors, which is overlooked in the traditional model, as their size was considered negligible compared to the main energy storage.
+The major source of this discrepancy is the buffering effect of the system's inherent capacitance, mostly coming from its decoupling capacitors.
+This aspect is overlooked in the traditional model, as the inherent capacitance was considered negligible compared to the main energy storage.
 
 Decoupling capacitors are on-board capacitors that act as energy buffers.
-They are mandatory components since the buffered energy prevent transient voltage drop when the system suddenly draws a large current such as during checkpointing (Sec.~\ref{sec:system_description}).
-However, their buffering effect introduces discrepancies between the execution model and the actual system behavior.
+They are mandatory components since the buffered energy prevent transient voltage drop when the system suddenly draws a large current, such as during checkpointing (Sec.~\ref{sec:system_description}).
+However, at the same time, their buffering effect introduces discrepancies between the execution model and the actual system behavior.
 For example, when the system powers on, decoupling capacitors are quickly charged using the energy in the storage, making the energy storage voltage an unreliable estimate of available energy.
 This buffered energy also allows the system operate for a while at a sub-normal voltage after the power supply is stopped.
-Additionally, between power cycles, decoupling capacitors discharge due to the resistance of the system, which significantly lowers the power efficiency.
-In the systems with small capacitors, these effects dominate the behaviors that modeled in the traditional model.
-Consequently, techniques that are highly efficient according to the traditional model may introduce substantial power overhead and even correctness issues in small-scale systems.
+Additionally, between power cycles, decoupling capacitors discharge due to the resistance of the system, considerably lowering the power efficiency.
+In the systems with small capacitors, these effects dominate the behaviors that are modeled in the traditional model.
+Consequently, highly efficient techniques according to the traditional model may introduce substantial power overhead and even correctness issues in small-scale systems.
 % Consequently, designing software techniques based on the traditional model brings significant power overhead and even correctness issues, even they are extremely efficient in the traditional model.
 % While this seems merely delaying the start and the end of the operations at first glance, we will show that it significantly affects the power efficiency and even correctness of software designs.
 
@@ -54,7 +56,8 @@ Consequently, techniques that are highly efficient according to the traditional
 % Then the buffered energy continues to power the system even after the power-off threshold is reached. 
 % Understanding this effect is crucial for intermittent system designers, as it has a significant impact on both system efficiency and correctness.
 
-In this paper, we propose a detailed execution model for intermittent systems including the effects of decoupling capacitors. 
-In Section 2, we demonstrate that understanding this model is critical for software designers. 
-Our evaluation shows that intermittent systems designed upon the traditional model can be up to six times more energy-inefficient and may fail to predict power-off timings accurately, leading to unsafe checkpointing. 
-Based on the insights from the detailed model, we propose design guidelines to implement efficient and safe intermittent systems.
+In this paper, we propose a new execution model for intermittent systems which includes the buffering effects of decoupling capacitors.
+In Sec.~\ref{sec:detailed_execution_model}, we demonstrate that understanding this model is critical for software designers:
+intermittent systems designed upon the traditional model can be up to 5.62x more energy-inefficient than expected and may fail to predict power-off timings accurately, leading to unsafe checkpointing. 
+In Sec.~\ref{sec:design_guidelines}, we propose design guidelines to implement efficient and safe intermittent systems with small energy storages, based on the insights from our model.
+Without incurring any extra overhead, our proposed power failure prediction methods improve end-to-end execution latency of both dynamic and static checkpointing schemes, by 2.86x and 3.04x on average, respectively.

sections/OurApproach.tex

@@ -1,7 +1,15 @@
 \section{Design Guidelines}
+\label{sec:design_guidelines}
 
 \subsection{Delay Checkpoint Execution}
 
+\begin{figure}
+    \centering
+    \includegraphics[width=\linewidth]{figs/plot_expr_7_cropped.pdf}
+    \caption{Execution times across various checkpoint voltages, normalized to the 3.4V case.}
+    % \label{fig:hardware_setup}
+\end{figure}
+
 \subsection{Use Vdd for Checkpoint Trigger}
 
 \begin{figure}

sections/OurModel.tex

@@ -1,8 +1,9 @@
 \section{Detailed Intermittent Execution Model}
+\label{sec:detailed_execution_model}
 
-In this Section, we provide a detailed description of our intermittent system execution model and its implications for software design.
-In Sec.~\ref{sec:system_description}, we introduce the target architecture of our model and the reference system used for evaluations.
-Sec.~\ref{sec:execution_model} presents our detailed execution model, designed based on the key observations from our experimental results.
+In this section, we describe our execution model and its implications for software design.
+In Sec.~\ref{sec:system_description}, we introduce target architecture and the reference system used for evaluations.
+Sec.~\ref{sec:execution_model} presents the proposed execution model, designed based on the key observations from experimental results.
 In the following three sections, we discuss how this model affects both the power efficiency and correctness of software design.
 Finally, in Sec.~\ref{sec:other_architectures}, we evaluate the effectiveness of our model across systems with different architectural configurations. 
 
@@ -26,10 +27,10 @@ Also, the computing system has its own decoupling capacitor (C2) to stabilize op
 
 Recent studies increasingly explore 32-bit architectures for the computing system~\cite{shihIntermittent2024,wuIntOS2024,kimRapid2024,akhunovEnabling2023,kimLACT2024,kimLivenessAware2023,parkEnergyHarvestingAware2023,kortbeekWARio2022,khanDaCapo2023,barjamiIntermittent2024,songTaDA2024}, as emerging applications on intermittent systems, such as Deep Neural Networks (DNNs)~\cite{houTale2024,yenKeep2023,khanDaCapo2023,gobieskiIntelligence2019,islamEnabling2022,kangMore2022,leeNeuro2019,islamZygarde2020,custodeFastInf2024,barjamiIntermittent2024,songTaDA2024}, demand greater computational capabilities~\cite{bakarProtean2023a,carontiFinegrained2023}.
 % Emerging intermittent applications demand increasing computing capability~\cite{bakarProtean2023a} due to computat
-In this context, we employ a custom-built board featuring a 32-bit ARM Cortex-M33 processor (operating at 16Mhz) with 512KB of FRAM as a reference system.
-For the power management system, we use a TI BQ25570 based system.
-We empirically select XXuF and 220uF capacitors for C1 and C2, respectively, as these are the minimum capacitor sizes for stable checkpointing and recovery.
-Sec.~\ref{sec:other_architectures} evaluates generality of our model in different architectures, such as systems equipped with MRAM and a 16-bit core.
+In this context, we employ a custom-built board featuring a 32-bit ARM Cortex-M33 processor (operating at 16Mhz) with 512KB of Ferroelectric RAM (FRAM) as a reference system.
+A TI BQ25570 based board is used for the power management system.
+We empirically select 22uF and 220uF capacitors for C1 and C2, respectively, as these are the minimum capacitor sizes for stable checkpoint and recovery.
+Sec.~\ref{sec:other_architectures} evaluates generality of our model in different architectures, such as systems with Magnetic RAM (MRAM) and a 16-bit core (e.g., MSP430).
 
 % In this work, our goal is to model the buffering effects of these capacitors and evaluate their implications on software designs.
 % (Recent studies present the need for better computing capability~\cite{bakarProtean2023a})
@@ -57,23 +58,22 @@ Sec.~\ref{sec:other_architectures} evaluates generality of our model in differen
     \label{fig:execution_trace}
 \end{figure}
 
-To derive general execution model with the effects of decoupling capacitors, we first present a sample measurement in our reference system.
-To generate operation time of 50ms under 1.5mA current supply, we use a 470uF capacitor for the energy storage.
-Fig.~\ref{fig:execution_trace_one_cycle} shows the voltage trace of the energy storage and the operating voltage (Vdd) of the computing system for one power cycle.
+To derive general execution model with the effects of decoupling capacitors, we first present a sample measurement from our reference system.
+To generate operation time of 50ms under 1.5mA current supply, we use a 470uF capacitor for energy storage.
+Fig.~\ref{fig:execution_trace_one_cycle} shows the traces of the energy storage voltage and the MCU operating voltage (Vdd) for one power cycle.
 Note that Vdd is maintained by decoupling capacitors after current supply from the power management system stops.
 The shaded areas represent the ranges that system executes the application code.
 % Fig.~\ref{fig:execution_trace_one_cycle} shows the trace during one power cycle, and Fig.~\ref{fig:execution_trace_detailed} presents the first execution cycle in more detail.
 
-Fig.~\ref{fig:execution_trace_detailed} presents the first execution cycle in detail. It shows several interesting differences between the traditional execution model and the actual operation.
+Fig.~\ref{fig:execution_trace_detailed} presents the first execution cycle in more detail. It shows several interesting differences between the traditional execution model and the actual operation.
 Among them, we highlight three key observations that affect software designer's decision.
 
 \begin{itemize}
     \item \textbf{O1}: The capacitor voltage drops quickly to charge decoupling capacitor when the system wakes-up ($t1$--$t2$).
     \item \textbf{O2}: The system executes at sub-voltage using the decoupling capacitors, even after power supply stops ($t4$--$t5$).
-    \item \textbf{O3}: The decoupling capacitor discharges while the system is powered-off (after $t5$).
+    \item \textbf{O3}: Decoupling capacitors discharge while the system is powered-off (after $t5$, as shown in Fig.~\ref{fig:execution_trace_one_cycle}).
 \end{itemize}
 
-
 \begin{figure}
     \centering
     \includegraphics[width=\linewidth]{figs/cropped/detailed_execution_model.pdf}
@@ -84,20 +84,22 @@ Among them, we highlight three key observations that affect software designer's
 % As we discuss in the following sections, all three observations significantly impact the performance of intermittent system designs.
 % We propose a detailed execution model which reflects these observations.
 Fig.~\ref{fig:detailed_execution_model} shows our detailed execution model, which reflects all the key observations.
-When the capacitor voltage reaches the power-on threshold, the voltage experience quick drop due to the buffering effects, instead of gradual reduction.
-After initialization, the system starts to execute at normal voltage (e.g., 3.3V).
-When the voltage hits the power-off threshold, the power supply stops but system now starts to execute using the buffered energy.
-Since voltage of capacitor decreases as it discharges, the system executes at sub-normal voltage until it reaches the voltage it cannot operate (e.g., 1.8V, typically known as Brown-Out Reset (BOR) voltage).
-Finally, until the next power-on, the remaining energy in decoupling capacitors continues to discharge.
-
-When designing intermittent systems with small capacitors, it is important for software designers to understand this model.
+When the capacitor voltage reaches the power-on threshold, the voltage experience quick drop due to the buffering effects (\circled{1}), instead of gradual reduction.
+After initialization (\circled{2}), the system starts to execute at normal voltage (\circled{3}), 3.3V for example.
+When the voltage hits the power-off threshold, the power supply stops but system now starts to execute using the buffered energy (\circled{4}).
+Since voltage of the decoupling capacitor decreases as it discharges, the system executes at sub-normal voltage until it reaches the voltage it cannot operate (e.g., 1.7V).
+% This voltage is known as Brown-Out Reset (BOR) voltage and is typically in a range of 1.7V to 2.5V in modern MCUs~\cite{}.
+Finally, until the next power-on, the remaining energy in decoupling capacitors continues to discharge (\circled{5}).
+
+When designing intermittent systems, especially targeting small capacitors, it is important for software designers to understand this model.
 In the following sections, we discuss the impact of this model to software design in more detail.
 
 \subsection{Impact on Power Efficiency}
+\label{sec:power_efficiency}
 
-The traditional model implies that power consumed between power-on and power-off thresholds are entirely used for the computing system.
+The traditional model implies that the energy consumed between power-on and power-off thresholds are entirely used for the computing system.
 However, our model reveals that considerable energy is used for charging the decoupling capacitors (\textbf{O1}) and dissipated during power-off durations (\textbf{O3}).
-This implies that much smaller energy is used for the useful computation compared to the designer's expectation when using small capacitors.
+This implies that much smaller energy may be used for the useful computation compared to the designer's expectation.
 
 \begin{figure}
     \centering
@@ -106,52 +108,83 @@ This implies that much smaller energy is used for the useful computation compare
     \label{fig:power_distribution}
 \end{figure}
 
-Fig.~\ref{fig:power_distribution} shows the distribution of the energy consumed for each stage of operation within one power cycle in various capacitor sizes, averaged over 50 executions. 
-The line in the secondary axis represents the average operation times for application code.
-The checkpoint is executed by the interrupt from the power management system, which is generated when the capacitor voltage reaches the power-off threshold (3.4V).
-Note that this is the last point for checkpoint execution according to the traditional model.
-
-The results shows that significant energy is wasted in decoupling capacitor.
-The cost is more expensive when the capacitor size is small since the discharge rate follows the RC-discharging circuits.
-While using smaller capacitors shortens the power-off durations, the discharging behavior penalizes them most since RC-discharging circuits discharge exponential (in our case, 50\% of energy is discharged at the first 161 ms).
-As a result, 60.7\% of power is wasted in 470uF, and the rate decreases as the capacitor size increases, down to 28.5\% in 1320uF case.
-
-More importantly, this wasted energy is expected to be used for the computation in traditional execution model, as all the energy except for the initialization and checkpoint/recovery is expected to be used in computations.
-It brings significant errors between the two models in available energy for the execution.
-In 470uF case, the actual energy efficiency (Execution) and the expectation from the traditional model (Execution and Discharged) differs by 4.99 times.
-
-(Limitations of power failure injection and simulation based evaluations).
+Fig.~\ref{fig:power_distribution} shows the distribution of the energy consumed for each stage of operation within one power cycle, averaged over 50 executions.
+An 1mA of input current is provided at 1.9V.
+The x-axis represents different capacitor sizes and the line in the secondary axis represents the average operation times for application code.
+The checkpoint is executed by the interrupt from the power management system~\cite{}, which is generated when the capacitor voltage reaches the power-off threshold (3.4V).
+Note that this is the most efficient point for checkpoint execution according to the traditional model.
+
+The results shows that significant energy is wasted in the decoupling capacitors.
+For example, 60.7\% of power is wasted during the power-off duration (denoted as \emph{Dischrged}) in 470uF case.
+The discharging behavior can be modeled as RC-discharging circuits (i.e., $q=CVe^{-\frac{1}{RC}t}$), which show exponential discharge rate.
+As a result, the cost from discharging is more expensive when the capacitor size is small;
+in our case, 50\% of energy is discharged at the first 161 ms.
+The discharge rate decreases as the capacitor size increases, down to 28.5\% in 1320uF case, which is still not negligible.
+% The cost is more expensive when the capacitor size is small since the discharge rate follows the RC-discharging circuits.
+% While using smaller capacitors shortens the power-off durations, the discharging behavior penalizes them most since RC-discharging circuits discharge exponentially (in our case, 50\% of energy is discharged at the first 161 ms).
+% As a result, 60.7\% of power is wasted in 470uF, and the rate decreases as the capacitor size increases, down to 28.5\% in 1320uF case.
+
+Another important observation is the error introduced by the traditional model.
+The traditional model expects both the energies, \emph{Execution} and \emph{Discharged}, are used for computation.
+This introduces significant errors, up to 5.62x in 470uF setup.
+In the same context, the traditional model expects using 470uF capacitor instead of 1320uF results in merely 1.22x overhead in energy efficiency, but the actual energy efficiency differs by 4.71x.
+% However, our model shows that the actual energy efficiency differs by xx\% in reality, brining xx\% error in the traditional model.
+This can significantly mislead the system designers when they decide the capacitor size by considering tradeoffs between overall efficiency and reactiveness.
+In Sec.~\ref{sec:design_guidelines}, we discuss our guidelines to minimize overhead from discharging when designing software techniques.
+
+% More importantly, this wasted energy is expected to be used for the computation in traditional execution model, as all the energy except for the initialization and checkpoint/recovery is expected to be used in computations.
+% It brings significant errors between the two models in available energy for the execution.
+% In 470uF case, the actual energy efficiency (Execution) and the expectation from the traditional model (Execution and Discharged) differs by 4.99 times.
+
+% (Limitations of power failure injection and simulation based evaluations).
+
+% In Sec.~\ref{sec:design_guidelines}, we discuss our guidelines to maximize power efficiency with software-level designs.
 
 \subsection{Impact on Predicting Power Failures}
 
+According to the traditional model, the system states should be saved to NVM before power-off threshold, as the system halts at this point.
+On the other hand, our model shows that the system may operate afterward using the energy stored in the decoupling capacitors (\textbf{O2}). 
+Modern MCUs can operate on a range of supply voltages (e.g., from 1.7V to 3.6V for STM32L5 and MSP430).
+Since the voltage of decoupling capacitors decreases as the discharge, the computing system is executed until the voltage reaches the minimum operating voltage.
+% While the voltage of decoupling capacitors decreases as they discharge, the computing system operates since modern MCUs can operate on a range of supply voltages (e.g., from 1.7V to 3.6V for STM32L5 and MSP430).
+This makes the energy storage voltage not a good estimate of the remaining time that system can execute.
+
 \begin{figure}
     \centering
     \begin{subfigure}{\linewidth}
         \includegraphics[width=\textwidth]{figs/plot_expr_6a_cropped.pdf}
-        \caption{Input current is 1mA.}
-        % \label{fig:eval_voltage_trace}
+        \caption{Input current = 1mA.}
+        \label{fig:sub_voltage_execution_1mA}
         \vspace{5pt}
     \end{subfigure}
     \begin{subfigure}{\linewidth}
         \includegraphics[width=\textwidth]{figs/plot_expr_6b_cropped.pdf}
-        \caption{Input current is 3mA.}
-        % \label{fig:eval_adaptivenss_finished_tasks}
+        \caption{Input current = 3mA.}
+        \label{fig:sub_voltage_execution_3mA}
     \end{subfigure}
     \caption{Ratio of sub-voltage operations in total execution time.}
     \label{fig:sub_voltage_execution}
 \end{figure}
 
-According to the traditional model, the system states should be saved to NVM before power-off threshold.
-Our model shows that the system may operate after this point using the energy stored in the decoupling capacitors (\textbf{O2}). 
-As a result, the energy storage voltage is not a good approximate of the remaining time that system can execute.
+% Modern MCUs can operate on wide range of operating voltages (e.g., from 1.7V to 3.6V for STM32L5 and MSP430).
+
+Fig.~\ref{fig:sub_voltage_execution} shows the ratio of the times executed under sub-voltage over the total execution times, averaged over 30 measurements.
+The x-axis shows the different capacitor sizes and the colors represent the voltages that system enters sleep state.
+We evaluate various voltages ranging from 1.7V to 2.5V since not all components in the computing system may operate at the lowest voltage (Sec.~\ref{sec:sub_normal_execution}).
+Also, we present two different cases with input current of 1mA (Fig.~\ref{fig:sub_voltage_execution_1mA}) and 3mA (Fig.~\ref{fig:sub_voltage_execution_3mA}) to evaluate the impact of input power.
 
-Modern MCUs can operate on wide range of operating voltages. 
-Our case: 1.7V to 3.6V.
+The figure shows that significant MCU operation is executed under sub-normal voltage.
+For example, when 470uF capacitor is used at 1mA input current (Fig.~\ref{fig:sub_voltage_execution_1mA}), 82.8\% of computation is executed \emph{after} power-off threshold.
+The ratio decreases as the system powers-off early (reduced sub-voltage operation time) or the input current increases (longer operation time at normal voltage).
+Under 1000uF is the major focus of this paper.
 
-Fig.~\ref{fig:sub_voltage_execution} shows the ratio of sub-voltage executions over total execution times.
-The x-axis shows the different capacitor sizes and the colors represent various power-off voltages.
+These values can be directly translated to the inefficiency of the system based on the traditional model.
+For example, in 470uF with 1mA input current case, systems executing checkpoint at power-off threshold execute 16.3 ms while it can operate 29.4 ms more if it execute checkpoint at 2.5V.
+Although executing checkpoint early may save some energy in decoupling capacitors, the saved energy is not preserved as discussed in Sec.~\ref{sec:power_efficiency}.
+In Sec.~\ref{sec:design_guidelines}, we validate this aspect and propose a method to execute checkpoint truly just before the poweroff.   
 
 \subsection{Impact of Sub-normal Voltage Execution}
+\label{sec:sub_normal_execution}
 
 The traditional model makes the software designers assume the system is executed under stable voltage.
 However, the execution after the power-off threshold (\textbf{O3}) happens in sub-normal voltage.