Nicolás Wolovick, 20200316
"La computación es la ciencia de la abstracción".
¿Mapeo no inyectivo R -> cantidad finita de bits?
123.45 -> 0001 0010 0011 0100 0101
123.45 -> 12345/100
123.45 -> 11000000111001
123.45 -> 0.12345 * 10^3
Si hacemos arqueología computacional, vamos a encontrar de todo.
Hay cosas curiosas, la ZX81 sólo manejaba flotantes cuando su µP sumaba enteros de hasta 16 bits.
(-1)^s * (1+mantisa) * 2^{exponente-offset}
Verdades de Pedro Grullo:
a+b = a, b!=0
1 REAL*4 X,Y
2 X = 1.25E8
3 Y = X + 7.5E-3
4 IF (X .EQ. Y) THEN
5 PRINT *,'Am I nuts or what?'
6 ENDIF
7 PRINT *,X,Y
8 END
0.1*10 != 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1
1 REAL*4 X,Y
2 X = 0.1
3 Y = 0
4 DO I=1,10
5 Y = Y+X
6 ENDDO
7 IF (Y .EQ. 1.0) THEN
8 PRINT *,'Algebra is truth'
9 ELSE
10 PRINT *,'Not here'
11 ENDIF
12 PRINT *,1.0-Y
13 END
x^-1 * x != 1
1 INTEGER X
2 DO X=1, 1000
3 IF ((1.0/X)*X .NE. 1.0) THEN
4 PRINT *, "X^-1*X != 1 for X=", X
5 ENDIF
6 ENDDO
7 END
Imprime 135 números.
1 (X+Y)+Z = (.00005 + .00005) + 1.0000
2 = .0001 + 1.0000
3 = 1.0001
4 X+(Y+Z) = .00005 + (.00005 + 1.0000)
5 = .00005 + 1.0000
6 = 1.0000
El no-determinismo en el orden de la suma se pierde.
No es un conjunto es una secuencia.
Secuencialidad => Corrección
¿Adiós paralelismo!
El estándar de "C"
dice que (x+y)+z
se puede evaluar como x+(y+z)
, pero en el C Standard 5.1.2.3 Program execution
previene diferentes evaluaciones si dan diferentes resultados.
Transformar divisiones en multiplicaciones siempre que se pueda.
1 float kahan(void) {
2 float s = x[0];
3 float c = 0.0f;
4 for (int i = 1; i < N; i++) {
5 float y = x[i] - c;
6 float t = s + y;
7 c = (t - s) - y;
8 s = t;
9 }
10 return s;
11 }
fp64
usando fp32
.-O3 -ffast-math
: caga la fruta.John D. Cook, Summing an array of floating point numbers, 20191105.
También define el "como" de las computaciones:
Hay no-determinismo en esta especificación
Esto permite diferentes implementaciones.
Implementaciones: ARM, Intel, Cray, NVIDIA, AMD, Xilnix, IBM, ...
grs
)grs
Todas las implementaciones permiten modificar el rounding mode.
Karen Miller, Floating Point Representation, 2006.
000...0
.NaN
es una constante que se propaga infinitamente.fp32
vs. fp64
fp64
ocupa el doble de memoria.fp64
.fp16
, fp24
y no IEEE754 compatible.fp16
para DNN y están encastrados en los Tensor Cores de Volta.fp80
y por fuera fp32
o fp64
.fp16
, fp32
, fp64
en NVIDIARyan Smith, The NVIDIA GeForce GTX 1080 & GTX 1070 Founders Editions Review: Kicking Off the FinFET Generation, Anandtech, July 2016.
fp64
, la jactancia de los ricosMatsuoka et.al, Double-precision FPUs in High-PerformanceComputing: an Embarrassment of Riches?, 2019.
A snail in the coffin for Linpack and its stunt machines.
By studying a large number of HPC proxy application, we found no significant performance difference between these two processors, despite one having more double-precision compute than the other.
Mixed precision algorithms FTW!
¡Todos los paquetes de MD!
bfloat16
We have developed an alternate approach to making AI models run efficiently. Building on a lineage of ideas reaching back to the early days of computer science more than 70 years ago, our method optimizes floating point itself.
Daniel Lemire
How many floating-point numbers are in the interval [0,1]?, 2017.
-- no me convence, no pude hacer el experimento que yo quería, lo dejo para que alguien lo mire --
--use_fast_math (-use_fast_math)
Make use of fast math library. -use_fast_math implies -ftz=true
-prec-div=false -prec-sqrt=false.
--ftz [true,false] (-ftz)
When performing single-precision floating-point operations, flush
denormal values to zero or preserve denormal values. -use_fast_math
implies --ftz=true.
Default value: 0.
--prec-div [true,false] (-prec-div)
For single-precision floating-point division and reciprocals, use IEEE
round-to-nearest mode or use a faster approximation. -use_fast_math
implies --prec-div=false.
Default value: 1.
--prec-sqrt [true,false] (-prec-sqrt)
For single-precision floating-point square root, use IEEE
round-to-nearest mode or use a faster approximation. -use_fast_math
implies --prec-sqrt=false.
Default value: 1.
--fmad [true,false] (-fmad)
Enables (disables) the contraction of floating-point multiplies and
adds/subtracts into floating-point multiply-add operations (FMAD, FFMA,
or DFMA). This option is supported only when '--gpu-architecture' is
set with compute_20, sm_20, or higher. For other architecture classes,
the contraction is always enabled. -use_fast_math implies --fmad=true.
Default value: 1.
-ffast-math
Sets -fno-math-errno, -funsafe-math-optimizations, -ffinite-math-only, -fno-rounding-math, -fno-signaling-nans and
-fcx-limited-range.
-funsafe-math-optimizations
Allow optimizations for floating-point arithmetic that (a) assume that arguments and results are valid and (b) may violate IEEE
or ANSI standards. When used at link-time, it may include libraries or startup files that change the default FPU control word
or other similar optimizations.
This option is not turned on by any -O option since it can result in incorrect output for programs that depend on an exact
implementation of IEEE or ISO rules/specifications for math functions. It may, however, yield faster code for programs that do
not require the guarantees of these specifications. Enables -fno-signed-zeros, -fno-trapping-math, -fassociative-math and
-freciprocal-math.
The default is -fno-unsafe-math-optimizations.
Modificar el registro de estado MXCSR
.
FZ bit 15 Flush To Zero
R+ bit 14 Round Positive
R- bit 13 Round Negative
RZ bits 13 and 14 Round To Zero
RN bits 13 and 14 are 0 Round To Nearest
Modificadores de las instrucciones de assembler:
.rn Round to nearest even.
.rz Round towards zero.
.rm Round towards -infty.
.rp Round towards +infty.
fp64
de manera juiciosa (ancho de banda, memoria).fp64
y almacena los vectores en fp32
.John Gustafson, A Radical Approach to Computation with Real Numbers, JSFI, 2016.
Naturales: notación binaria
Enteros: representación complemento a 2s.
1 8: char
2 16: short
3 32: int, long
4 64: long long
¡Depende de la arquitectura!
64 bits
1 8: char
2 16: short
3 32: int
4 64: long
1 #include <inttypes.h>
2
3 int8_t spin;
4 uint64_t total_sum;
R. Munroe, INT(PI), XKCD 1275.
Table of Contents | t |
---|---|
Exposé | ESC |
Full screen slides | e |
Presenter View | p |
Source Files | s |
Slide Numbers | n |
Toggle screen blanking | b |
Show/hide slide context | c |
Notes | 2 |
Help | h |