Home > Community > Blogs > System Design and Verification > understanding latency vs throughput
 
Login with a Cadence account.
Not a member yet?
Create a permanent login account to make interactions with Cadence more conveniennt.

Register | Membership benefits
Get email delivery of the System Design and Verification blog (individual posts).
 

Email

* Required Fields

Recipients email * (separate multiple addresses with commas)

Your name *

Your email *

Message *

Contact Us

* Required Fields
First Name *

Last Name *

Email *

Company / Institution *

Comments: *

Understanding Latency versus Throughput

Comments(3)Filed under: High-Level Synthesis, ESL, ESL High Level Synthesis, Sergio Ramirez, latency, throughput, Team ESLOne of the effects of adopting a High Level Synthesis design methodology is that the barrier between "Systems designers" and "Hardware designers" is substantially reduced if not totally eliminated. Suddenly, both "Systems designers" and "Hardware designers" are using not only the same input language to specify their models (C++ / System C) but they are also exposed to the same terminology. For this reason, "Hardware designers" are suddenly exposed to two terms to which they have had little or no exposure in the past.

The purpose of this post is to clarify two "systems" terms that are usually confused and sometimes used interchangeably: latency and throughput.

Definition of terms

Let us attempt to define those two terms:

Latency is the time required to perform some action or to produce some result. Latency is measured in units of time -- hours, minutes, seconds, nanoseconds or clock periods.

Throughput is the number of such actions executed or results produced per unit of time. This is measured in units of whatever is being produced (cars, motorcycles, I/O samples, memory words, iterations) per unit of time. The term "memory bandwidth" is sometimes used to specify the throughput of memory systems.

A simple example

The following manufacturing example should clarify these two concepts:

An assembly line is manufacturing cars. It takes eight hours to manufacture a car and that the factory produces one hundred and twenty cars per day.

The latency is: 8 hours.

The throughput is: 120 cars / day or 5 cars / hour.

A design example

Now that these two concepts are clear, let us apply these concepts to a problem "closer to home." 

A designer is given the task to create hardware for a communications device that has the following characteristics:

Clock frequency:                                          100MHz

Time available to perform the computation:     1000ns

Throughput of the device:                              640 Mbits / second

Word width of each output:                            64 bits

Let us translate these requirements into latency and throughput measurements that are more meaningful from the point of view of the hardware designer.

Latency: 1000 ns = 1000 ns * (1 s / 10^9 ns ) * ( 100 * 10^6 clock periods/ 1s) = 10^11/10^9 = 100 clock periods.

Throughput = 640 Mbits / s = (640 * 10^6 bits/s) * (1 word / 64 bits) * ( 1 s / 100 * 10^6 clock periods) =  640 * 10^6 / 64 * 100 * 10^6 = 10 * 10 / 100 = 1 / 10 = 0.1 words / clock period.

The throughput could be read more conveniently as follows: "one word every 10 clock periods"

Latency expressed in clock periods, and throughput expressed in number of available clock cycles between words, are parameters that a designer can use to create the desired hardware according to the performance specifications.

A final clarification

Some tools do not express the throughput in units per unit of time but in clock periods. This is incorrect but commonly used because of convenience. Therefore some tools would report the throughput of our communication algorithm as 10.

This Team ESL posting is provided by Dr. Sergio Ramirez, Sr Staff Product Engineer for the C-to-Silicon Compiler high level synthesis product.

Comments(3)

By me on October 21, 2010
wow that's kind of insulting. I seen more systems "engineers" who fail to grasp the difference than h/w "designers".

By Dana French on November 18, 2011
An extrapolation on the example in this article:
An assembly line is manufacturing cars. It takes eight hours to manufacture a car and that the factory produces one hundred and twenty cars per day.
The latency is: 8 hours.
The throughput is: 120 cars / day or 5 cars / hour.
----
What if I build another identical assembly line to manufacture more cars? It still takes eight hours to manufacture a care on this new assembly line, however now I have two lines with twice as many employees, so my throughput is doubled.
The latency is: 8 hours.
The throughput is: 120 cars / line / day or 10 cars / hour.
This example shows that doubling the assembly line facilities (hardware) and doubling the number of employees (software) results in doubling the throughput.
----
What if I can reduce the speed of both assembly lines by 50% and use the employees from the first assembly line to work both lines?
It would now take 12 hours to manufacture each car and each line would produce 80 cars per day.
The latency is: 12 hours.
The throughput is: 80 cars / line / day or 6.67 cars / hour.
This example shows that doubling the assembly line facilities (hardware) but keeping the number of employees (software) constant, results in increased throughput, even though the latency is higher.

By harita kambhampati on May 24, 2012
thank you for discussing such a wonderful topic

Leave a Comment


Name
E-mail (will not be published)
Comment
 I have read and agree to the Terms of use and Community Guidelines.
Community Guidelines
The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.