From 1cd5978fb763b5e5b5bf637c1743d729d813110f Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Oct 2021 11:25:56 -0700 Subject: [PATCH 001/277] Add printf FAQ --- docs/source/faq.rst | 48 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 4a953d973..45bb7894a 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -40,6 +40,7 @@ Questions answered in this FAQ: #. :ref:`Why are compressed arrays so slow? ` #. :ref:`Do compressed arrays use reference counting? ` #. :ref:`How large a buffer is needed for compressed storage? ` + #. :ref:`How can I print array values? ` ------------------------------------------------------------------------------- @@ -1143,3 +1144,50 @@ table lists the maximum block size (in bits) for each scalar type, whether | double +---------+-------+-------+-------+-------+ | | |check| | 278 | 1058 | 4178 | 16658 | +--------+---------+-------+-------+-------+-------+ + +------------------------------------------------------------------------------- + +.. _q-printf: + +Q29: *How can I print array values?* + +Consider the following seemingly reasonable piece of code:: + + #include + #include "zfparray1.h" + + int main() + { + zfp::array1 a(100, 16.0); + printf("%f\n", a[0]); // does not compile + return 0; + } + +The compiler will complain about :code:`a[0]` being a non-POD object. This +is because :code:`a[0]` is a :ref:`proxy reference ` object +rather than a :code:`double`. To make this work, :code:`a[0]` must be +explicitly converted to :code:`double`, e.g., using a cast:: + + printf("%f\n", (double)a[0]); + +For similar reasons, one may not use :code:`scanf` to initialize the value +of :code:`a[0]` because :code:`&a[0]` is a :ref:`proxy pointer ` +object, not a :code:`double*`. Rather, one must use a temporary variable, +e.g. +:: + + double t; + scanf("%lf", &t); + a[0] = t; + +Note that using :code:`iostream`, expressions like +:: + + std::cout << a[0] << std::endl; + +do work, but +:: + + std::cin >> a[0]; + +does not. From bbfb88db897b95f4e3aaeab8d842f65c169295c4 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Oct 2021 17:07:43 -0700 Subject: [PATCH 002/277] Clarify FAQ #17 on violated tolerances --- docs/source/faq.rst | 60 ++++++++++++++++++++++++++++++++------------- 1 file changed, 43 insertions(+), 17 deletions(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 45bb7894a..c97a4c655 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -552,17 +552,40 @@ Q17: *Why does zfp sometimes not respect my error tolerance?* A: First, |zfp| does not support :ref:`fixed-accuracy mode ` for integer data and will ignore any tolerance requested via :c:func:`zfp_stream_set_accuracy` -or associated :ref:`expert mode ` parameter settings. - -For floating-point data, |zfp| does not store each scalar value independently -but represents a group of values (4, 16, 64, or 256 values, depending on -dimensionality) as linear combinations like averages by evaluating arithmetic -expressions. Just like in uncompressed IEEE floating-point arithmetic, both -representation error and roundoff error in the least significant bit(s) often -occur. +or associated :ref:`expert mode ` parameter settings. So this +FAQ pertains to floating-point data only. + +The short answer is that, given finite precision, the |zfp| and IEEE +floating-point number systems represent distinct subsets of the reals +(or, in case of |zfp|, blocks of reals). Although these subsets have +significant overlap, they are not equal. Consequently, there are some +combinations of floating-point values that |zfp| cannot represent exactly; +conversely, there are some |zfp| blocks that cannot be represented exactly +as IEEE floating point. If the user-specified tolerance is smaller than the +difference between the IEEE floating-point representation to be compressed +and its closest |zfp| representation, then the tolerance necessarily will +be violated (except in :ref:`reversible mode `). In +practice, absolute tolerances have to be extremely small relative to the +numbers being compressed for this issue to occur, however. + +Note that this issue is not particular to |zfp| but occurs in the conversion +between any two number systems of equal precision; we may just as well fault +IEEE floating point for not being able to represent all |zfp| blocks +accurately enough! By analogy, not all 32-bit integers can be represented +exactly in 32-bit floating point. The integer 123456789 is one example; the +closest float is 123456792. And, obviously, not all floats (e.g., 0.5) can +be represented exactly as integers. + +To further demonstrate this point, let us consider a concrete example. |zfp| +does not store each floating-point scalar value independently but represents +a group of values (4, 16, 64, or 256 values, depending on dimensionality) as +linear combinations like averages by evaluating arithmetic expressions. +Just like in uncompressed IEEE floating-point arithmetic, both representation +error and roundoff error in the least significant bit(s) often occur. To illustrate this, consider compressing the following 1D array of four -floats:: +floats +:: float f[4] = { 1, 1e-1, 1e-2, 1e-3 }; @@ -580,20 +603,23 @@ is even smaller: 5.424e-9. This reconstruction error is primarily due to |zfp|'s block-floating-point representation, which expresses the four values in a block relative to a single, common binary exponent. Such exponent alignment occurs also in regular IEEE floating-point operations like addition. -For instance,:: +For instance, +:: float x = (f[0] + f[3]) - 1; should of course result in :code:`x = f[3] = 1e-3`, but due to exponent alignment a few of the least significant bits of f[3] are lost in the -addition, giving a result of :code:`x = 1.0000467e-3` and a roundoff error -of 4.668e-8. Similarly,:: +rounded result of the addition, giving :code:`x = 1.0000467e-3` and a +roundoff error of 4.668e-8. Similarly, +:: float sum = f[0] + f[1] + f[2] + f[3]; should return :code:`sum = 1.111`, but is computed as 1.1110000610. Moreover, the value 1.111 cannot even be represented exactly in (radix-2) floating-point; -the closest float is 1.1109999. Thus the computed error:: +the closest float is 1.1109999. Thus the computed error +:: float error = sum - 1.111f; @@ -601,7 +627,7 @@ which itself has some roundoff error, is 1.192e-7. *Phew*! Note how the error introduced by |zfp| (5.472e-9) is in fact one to two orders of magnitude smaller than the roundoff errors (4.668e-8 and -1.192e-7) introduced by IEEE floating-point in these computations. This lower +1.192e-7) introduced by IEEE floating point in these computations. This lower error is in part due to |zfp|'s use of 30-bit significands compared to IEEE's 24-bit single-precision significands. Note that data sets with a large dynamic range, e.g., where adjacent values differ a lot in magnitude, are more @@ -611,9 +637,9 @@ The moral of the story is that error tolerances smaller than machine epsilon (relative to the data range) cannot always be satisfied by |zfp|. Nor are such tolerances necessarily meaningful for representing floating-point data that originated in floating-point arithmetic expressions, since accumulated -roundoff errors are likely to swamp compression errors. Because such roundoff -errors occur frequently in floating-point arithmetic, insisting on lossless -compression on the grounds of accuracy is tenuous at best. +roundoff errors are likely to swamp compression errors. Because such +roundoff errors occur frequently in floating-point arithmetic, insisting on +lossless compression on the grounds of accuracy is tenuous at best. ------------------------------------------------------------------------------- From 57c50e038ed944eb01c11d5a7a6ead63bb558c25 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Oct 2021 18:30:17 -0700 Subject: [PATCH 003/277] Add FAQ on error distributions --- docs/source/faq.rst | 56 +++++++++++++++++++++++++++++++++-- docs/source/installation.rst | 16 +++++----- docs/source/zfp-rounding.pdf | Bin 0 -> 48460 bytes 3 files changed, 62 insertions(+), 10 deletions(-) create mode 100644 docs/source/zfp-rounding.pdf diff --git a/docs/source/faq.rst b/docs/source/faq.rst index c97a4c655..d44a87b15 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -41,6 +41,7 @@ Questions answered in this FAQ: #. :ref:`Do compressed arrays use reference counting? ` #. :ref:`How large a buffer is needed for compressed storage? ` #. :ref:`How can I print array values? ` + #. :ref:`What is known about zfp compression errors? ` ------------------------------------------------------------------------------- @@ -1003,9 +1004,10 @@ for 3D data, while the difference is smaller for 2D and 1D data. We recommend experimenting with tolerances and evaluating what error levels are appropriate for each application, e.g., by starting with a low, conservative tolerance and successively doubling it. The distribution of -errors produced by |zfp| is approximately Gaussian, so even if the maximum -error may seem large at an individual grid point, most errors tend to be -much smaller and tightly clustered around zero. +errors produced by |zfp| is approximately Gaussian (see +:ref:`FAQ #30 `), so even if the maximum error may seem large at +an individual grid point, most errors tend to be much smaller and tightly +clustered around zero. ------------------------------------------------------------------------------- @@ -1217,3 +1219,51 @@ do work, but std::cin >> a[0]; does not. + +------------------------------------------------------------------------------- + +.. _q-err-dist: + +Q30: *What is known about zfp compression errors?* + +A: Significant effort has been spent on characterizing compression errors +resulting from |zfp|, as detailed in the following publications: + +* P. Lindstrom, + "`Error Distributions of Lossy Floating-Point Compressors `__," + JSM 2017 Proceedings. +* J. Diffenderfer, A. Fox, J. Hittinger, G. Sanders, P. Lindstrom, + "`Error Analysis of ZFP Compression for Floating-Point Data `__," + SIAM Journal on Scientific Computing, 2019. +* D. Hammerling, A. Baker, A. Pinard, P. Lindstrom, + "`A Collaborative Effort to Improve Lossy Compression Methods for Climate Data `__," + 5th International Workshop on Data Analysis and Reduction for Big Scientific Data, 2019. +* A. Fox, J. Diffenderfer, J. Hittinger, G. Sanders, P. Lindstrom. + "`Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods `__," + SIAM Journal on Scientific Computing, 2020. + +In short, |zfp| compression errors are roughly normally distributed as a +consequence of the central limit theorem, and can be bounded. Because the +error distribution is normal and because the worst-case error is often much +larger than errors observed in practice, it is common that measured errors +are far smaller than the absolute error tolerance specified in +:ref:`fixed-accuracy mode ` +(see :ref:`FAQ #22 `). + +It is known that |zfp| errors can be slightly biased and correlated (see +:numref:`zfp-rounding` and the third paper above). Recent work has been +done to combat such issues by supporting optional +:ref:`rounding modes `. + +.. _zfp-rounding: +.. figure:: zfp-rounding.pdf + :figwidth: 90 % + :align: center + :alt: "zfp rounding modes" + + |zfp| errors are normally distributed. This figure illustrates the + agreement between theoretical (lines) and observed (dots) error + distributions (*X*, *Y*, *Z*, *W*) for 1D blocks. Without proper rounding + (left), errors are biased and depend on the relative location within a |zfp| + block, resulting in errors not centered on zero. With proper rounding + (right), errors are both smaller and unbiased. diff --git a/docs/source/installation.rst b/docs/source/installation.rst index e7ae4c2d4..852bde92c 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -252,16 +252,18 @@ in the same manner that :ref:`build targets ` are specified, e.g., CMake default: off. GNU make default: off and ignored. +.. _rounding: .. c:macro:: ZFP_ROUNDING_MODE **Experimental feature**. By default, |zfp| coefficients are truncated, - not rounded, which can result in biased errors. To counter this, two - rounding modes are available: :code:`ZFP_ROUND_FIRST` (round during - compression; analogous to mid-tread quantization) and :code:`ZFP_ROUND_LAST` - (round during decompression; analogous to mid-riser quantization). - With :code:`ZFP_ROUND_LAST`, the values returned on decompression are - slightly modified (and usually closer to the original values) without - impacting the compressed data itself. This rounding mode works with all + not rounded, which can result in biased errors (see + :ref:`FAQ #30 `). To counter this, two rounding modes are + available: :code:`ZFP_ROUND_FIRST` (round during compression; analogous + to mid-tread quantization) and :code:`ZFP_ROUND_LAST` (round during + decompression; analogous to mid-riser quantization). With + :code:`ZFP_ROUND_LAST`, the values returned on decompression are slightly + modified (and usually closer to the original values) without impacting the + compressed data itself. This rounding mode works with all :ref:`compression modes `. With :code:`ZFP_ROUND_FIRST`, the values are modified before compression, thus impacting the compressed stream. This rounding mode tends to be more diff --git a/docs/source/zfp-rounding.pdf b/docs/source/zfp-rounding.pdf new file mode 100644 index 0000000000000000000000000000000000000000..6cda53b3f4c8f1747ccb3e6705c814edab5aa92d GIT binary patch literal 48460 zcmafZbzD?k+b*fn4MR$I4c*;c;?OWOQqrB$14ttvAuWS2lyrAUNr}=SB_f@ktjrhgl#syW(Xe&3_i~`+ z7oq+42a`w1(b>z^gO*3h*~-gS!PeT%#uigb3e(fe!`8|L)9(w%7M}qxlAJucnI{9f z83QUq)RrlC@T93cdREW}8Ezyy{71ooPNXiazlKv+Spjf8s#2p-#5eUYI=k-cYZ zLcFws0=z)7|6A3crt5gPS?k++(V75FP*9@fF|hUXqUBL_0es8-dzJt9s!9u3%ey(d zdFZ=aS=-Y7F_-rQ(+dAvHc$o71PKYii2)z2$R89K3DAoEI})M=|7qJFP=przr#XLy z00#cs-alhLTJWF6`HKtCg8#0=9~?0K+gzZxJpcCaZ+KM~FfAYiEHHtb0qcKq0ml5a z;J+FC!2wTybNDkBq~-gY#Gf&+e1Eg}GZv=h`*)F5T|{X4{$>H(YJUR$dtdT_g@i@^ zzqcgv(OdO;#uk_3%X7BTI_JybA339Nc#yN;B{w|7#rqXmCRu!(E+n32EI;L_CNvZY z`So}R{#@~Z83eO(vPYFMl~E-kRN|^&A+-Hr_dKLy^~ark&hf!Z&V^0?uTAGSrau;* zcD6m7p0Czegr07C-CqnHl>ELwF@bE=tO_TauiWz7kFA9Myt!NmZF|r{54~$7GjAhB zZci&Xj2a6rxT^NI(z%OThT|;NoCdd2F9&x?IgGVxoqC6Gf80axWN8aJcC04t-nUqk zTenP{-Zx7%9bJ6QAM;+J`MBLIT_PUh`*!Cjr0ZeQyY;TfYh1W%hv;(me(`urIrQdS zcxLRzactASAnBRd_DvIu=BZbpME-$y%jCzQ`xevA`}}g6OGlH-#82kQ-d#2NIwdSS zyAsl^U1EyA0}4p*4+i}AK&8VL<-5U0{%_xfc4*Ib1fCkO!UT#vL(f75BJMPa`?g2? z?pqy}$QnZ>N%xL^sokHP+{=j<4 zclW&Dz#_}hB>(bOQED_qY4WZrGdPg<%~{(0?#;rkXK~t?#l8J4$9-$OlfUQD?x@La z!-d;@)!8pqijb*`Pg( z2FZYnXOcHRT~;E~982LLd`CLzL(iUuYVdW@j7uAY21vP1246nYVR?v(3%w`#{eq?I zJYaUA@wfCeYWawCz1Pu@PaaE`pgAc@fm%(lzyZzs8$|Mo%@l`<`L#cA9?x3kC*|9QxJqKx zK-zpIG8^svsg^yz@SO8Q!UHpH1J5|mL^&t2q!(JOQvAgvGk?ULi=J~X2ZwlRJCAkP zFEw@iu6>(y0P(StQ!ez~$^U6eNh;;sy85v)(597VGC%Iw{j~dt9Q%Z03-@fp1Pk}Y zo4{LaBKx;u+@xEMPD(mi4y0`#ZkHl2iO48t;$AH2eDXmn+x?o<-f-3;rEPlPMRDsG z%-zwcaRBx;C7b^>`%{~~GOJVC82z;mLh?+FpSr$6rr3eV$DmlK*PF^@QgC%?Ow5R% z;+y*(r(fQgWq#9c+>3k@O&QWxrL*~1o5BUnG?Qt})qv{U1^X(D8NK=4-cqIH2B`Jq!leNw^}N0htUc4?jjvs<9>|Q-XvMR3 zynwaJv6tE<&3>^F;jVDN9`%r#x$<$-YVt5MEGW z#p@C7T(|NqLJwU!*}q2{Cm;-%=BG3)&||*f-hQ_?dhRFXuVmcH|LGUDVdlse;W;#CXEDQ;q`gX0f!@w*$TK28 zn^|Ft#R1ZkDDM#B58GO^QFS#?dj5zp>PQ{dGnbUcT1K{o)>*$`rFk7=G0DroJ)E=K z52j<$0~M3C`f<#b2FAZR@!+aObjQuQRz1qecg0yI1OYiGPjXySi9p znhCDIj^2Bb&;K~$zE!$xrW}g}cUsi0SulJ;39TAZ8D(qs3=7mYW28Ya7}_O8KKTo6 zpNA9ch1|7(Bl)W#tJVShUn%%k-Mw{V(-Br*G?Gcp3ytL$(pi$8MHirZvaj}xa*HYP zOxXS`a2D+Ku)I)P79rT1lAiXyf9s+I0uf%{_bz)al6U{|Z35|%%nE<0Vc_<`!y4zL z*a+Pqo|77Jwom6dp^ui!H_y&t2W6xXWcIy!c*xsxBhRYVfS1@_7j^W}HoDzB#A{j* z+ZN5SC*T?$^=k3;rp-^y;svM|EPnUz%qKuheuH@W%W2bWV`!D(XO5&_v57b1Pi6c{ z3VzPC9_aUY^h#pNDIc=iQ9%$$F!b@h^;^~q#@&0hh>_==UL6V4$?Q@tgmt8`siwG6 z2Bn2@4O%yv;f{83Y?HM_Y-H@_Qn-xB&Jd0^+7UAw)3&#cPc##z`*H;y(wop%)?pUc}@lON+hfRCbnIpO7| zi8$!R{lr&VCp0f>GjZB(;)_U&9YrJrEDJkZlpa!%rYDp>Z^=Y2mW#G34YXl>vz069 zIqb@oUaf&D`n*z5bk=(tYw4E(^7oM$>8zs%gFr&Yh!LK;?5OuBi;DBsl;0}rd>9w` z;r#Si(-(H&4TFW(I{C7(9s`|y8}NW`IQMUYj7z3MiyeeHF6_=v@G;7h7 z)UcI~V9}5K<XvW$$})aQyyZy}cRrtJAg=l-Y4KBXSQG%Rr=21s`f%q)S}jBK1lIlF<);I5O^zAd z$Lq242gwwoX6ksYwX;vx487_@Cr3BD-qpS`h;K;$w$nerDDl!|XfNpHAyi%9CP)-i zOGK9NQo4-FS8ErGvuMw&JB$&_Li|0*0?H=5vFPCtT!RCB_ZF9YHh54=qC;IPsoWQ# zdS)>SZEa%jDXgVOXy&a%HJWA8Kj+S$IdQhTLGxO!LO+I*?F(k1Lk$_~we(cr0_EDa z3}WhEMbzGQCRBePZ?3<~Um39b%xU{Y$%g@yOCF_5i{-j-8B}hZH&OnpnuX*VN{N}> z5=vI_R6uf|fsjRJ2^Bt)MWXoB(=t2q0W%uI>o;=5rOh>R4*nBPOE8m5a#m06roVXE z+xZF{1xlhJ`{TU;dH_L}U0Q2y~#bRI@?A>}*s@5x;JgvEi4k z&rY*LjBpzZnbX0>t0Eri`y;>^_*tv=QUYECSg|F=ki<89KB>KdZT=NRL;tlpbTN!I z*Qds-ngoIZYJ$Xlf30OP{;^~g!@|Z3B+gUZiTWE0A`g}rXCuM$#BAzVx56`&F&@ta ztbv$*h8NQLKs>$LpWb8s#LltyRCs~o@dPq2C9sK%GfjXzgg6Enxx6J{MCpecI#hUR zc>F+~{6lea7!uCwk4Bd5N!~SuobUjQNO{TycgP{b1LdMc)8ul@Y3iQP52HUiiEWvs4VqIgdCe6 zs46S}`^saQXdL5L;#ABrn869fRwgg)k#xvGgH~-YUWOXTV!bn=gq05H@dMw6$A;aj zD`*!;m9wL^~kTg~K949WOGycU@Y-}pf< zxFqPkwR!1vP`g|cT|TO%c;>uFWIwLEB7sqNAc$hvM|tSIkF{q(eIiV z1{V)Njn8+4mt((K3gZk>naBw^fAvI=B&&BlzI2NW#J4=KwHnCV` zkVupqn#}0Q4?_CJR(y>UCacusFWAt@`7yZ$MBiOGoWkh-N?4s{`?iO(^|`}gqNVhIAcWdW8+Z{bpOU(K z=3nc701%%c>'OpqlxF^>MUC}kj@QCvp;rOo(i5QYu#RFS`6jfOqLq^#O_grk^( zC@WRr_EQNSOhH6LhasgRfO*5`*N8HbaM&|$G;3c076;$J5j8^aKJT^qQgBdzs0IZ*q+ zKM{+jfF|W+KH@WMp(3v$Rvygr=qY3eDrM)J*TMcLSvA?gFPUO|h6R5r2Jw-m$HPbn zHN6369D~_Fe~KBbxkg?icFfSp4_3B^A${M6A?kcKaR2ABgRQ=zeDl0weDklV$X~2H zdyV*E2BZrEei@Yqb9_l`;DK2{`~F`Sw%9gsCh{DT;~Tx=eYHD#^r1s@ToKm$eW z(*GCIDJjJfv6~e5E7(cL{Ey8ad(nZ20Vpdy1YTs|KHe+=;mmjJLHyD{OUSVIB1IFi z>jTOFd_r_ks5sI|sZQ-ts63KXK>G$8hIlMNWDaA8y@PikmbiEG_~p zAXOFd> z*0m`Di9|mo@>MAU^xy~Os>1FX5yzUD;nq;ewVTvSkO*^kRL4*^*eL2K7l(>r#kefL z;d3`Qrv>sGu~>)ZrZ-YkmxO50$ByzRB@ujM{J^)e`4c2_SUl_z z1jEKem~WG#4wvARmL|O>=wPrH@gJn+V|eiWlRJpO90=n>F$pj5+d+x#kHHA4qq>N~%|!TSg!nYWWVkRi(=I_aO6lEJ z_xfS{^S|2i$j-@8CwWDxg|Vg5zQ6tzyKaB`tzjy1a9*B*X+i)Ar@!PZ@o^$Sxxvah z62*^?zg&!j_$E{a-63`KJQi%O`^Bw! z5#6me@y#ammHL+oO76ieq*Ko0lUOjbs;=kI0OJqry_>m^C{@un zkoS2Fy8TRspAa=x!_DWUg!zClaU@;w!5^9Ff-HX0NTLTZ!1I%A^{IJN2k8#1w{S|U*F?@$|^ zfH1yCF@sKXaRWb?zsoMi;KuuNu(JZ2Z@eHe>Oce`U}B}0(A8N4Vl`yLjTcPcXmM+% zBIcOnEkll13{eh);ta?DpUZ{7X3k2w#DpCGEI#rN2F5>4rQ9a47W@b-kJE#C7&L`J zo>K|Ki^pGJ+!YrsN7k`LXl4d!K;mgkb498wxu`gDd{Gcp#+D>0akH$N&_MF_r!$}v_r3LlVFNBZyBsp3U`MgB|k;^ zNL52qYd;>6|dcS zKThsRQ5GltAIQr#cREf`z(Wsa+9;)J`a$ z*Z8a!HC6rH{1d9vyXP^364In^4Qx0G9dO92*qR9V9(k4B180hoy?|~|*ehFM_Qn4F z50#y;@;)dNc{ak`fO95F62eIsPfe7NZXMb0q*tdLk8E!gP=XBYR?gIS^aJhkjRpHi zz_5u%k~6jWC1BrFJSw)Afj24tc^Vc&PlTiv_6U=ONj#-vKV)LhM*>@9RL>50>1kO{ zikZ86+1D5QL!szAP^L%iQR!}xa^50U%RV3p@)_#m+r@*LlYvbPK$P*z-#ANx*_v4jut-i;d{?TJZU&KSjXK zSYr&n665ky4nZ)BGiLs(aL`B@zu*Q_2U{^!MVlr8cXp&ul8@T694#9l{;>)z&(?d(i<-+RuX zKH!+S|3|jj&pvQc8|^V#*se3Omripj`)K<-3VXDylmrj!gaq$}N>WTC<3Jy90Hm3h z*!F{{qn!T0#U`}k_>l*k0b_tOU`%6)wyloOv4;fQ>P!R?y#*-gNdO1JO{)q0|4Zs6 z0cQ$kkE{!RzGl^PrKqHb7roSG%*$zxC-6{-J^Fl~`JvBOI_UOW=-pmd$nC>T-*IJu zQnkQJ8$R`5=w_CqksIrXlFw(KzWSy(@a6exsI=Md$)`MoS)|(eix-sQ8#iDf9Q#}1 zWm$tI0@E2b=0i`@$!jH78QT|~C-u&Fqt+kNqe>j}CU)QryDsZ+B4;B<%JZe~(&G`7 zF@0PdN6q0+wR~b4Zj2w6P9|7g!CB>vyUicy_Qybx9-*E3D;0tJ{BGgnM#ATHLn8BDo0w! z$B(>=KJVupjlAux?&s}p`WAZ~YEw$!5YK|vv^hv2X}{dGs)AI`LM~GonUk zUC8b_V(}}cmxgc?ae6|^nnH@pvaFffezneG)VoHcAS1^~%F+*O@4AmdY+{ZUr>f(U zz8b~F`VM1?yQUM_c{~kb7^*N;_V@yA6S-1(|I;;lP}EW2AbK^OMpVa&ylsGA%+R52hZ7<&XeeroqdYp&U`2F4-0copALs2bmxMH5Lo zYsZW06VbCI%GFRt%6w+HbG&B>dOX)Un}YSMkWDTtW3ZJcoZDbuzPNlV72Ae3m*{17 z9n-rvBV`D)NX~BN+|LXx%Q`Bt4}C7Gq?F6Jqp4{Z-IxqlNwnHhsgd3ui!_|KFh1wx zWvypD@CKZ@US_L+`iH|2K|*ybI!gWA^WVtr>FG_iL>@V)F++qX! zeBalhqOvSUKk{rYVX@j2$Zud87>BYbwD>sdN>CX&ifilNKo*hqQmIvS{j*Dy%?YD3 zn99OEnWDfZPo8k!vr?xzFR>4vkA59I>dU{E4m51&bfi7h)BQViknIvBjLXL?g# zpQsw05R;^k5-0KLlMC~kwb9zRM#Z_qudR(%dc$BvX0`+QI!6snxsE**DZ|aYwBG(E z=4bIlpAu(imd*=YhbYyR`9iK9)wkiKMfpiKi_GkB5n{{sa(-7P&Jme7Z0fMSz!$`$ zY?B%&PNQx~aGZcD_A|X;eEi*187f`JG5Ww^Kf~6RRNFt552iNY+a#szC@8;|&29|YQt10Jc9OLns(oCH zr}5$kUJ=8Btjr7upUhlH0>wFetf3Wp3;soC?X z^rf^ULt}?ucNPa8*jVYzB63A5856SSLG)1al@T|U3&wK6yOT3U-%$5$Z66e*Piirs zhV1kxtKgqc7cpPIaVD3mWnp*D$!OvoLVFj0v78~9|F(Kt}*_1h&W__2=N(<@7G7k<~=NiU2ttlDImgkWvlB}mhj!!fYyp0z3HisUxB zuqKX0*&q<>1y(k~U$gn7Mv|sl7Cd&1jZrCUuVSDfm)9R`THjnM$Hd;wFYUmW+_L z-udw(-K?+zf`!M=cNr->5JUKZ{Z1@XhEzh)&d&2Tp8NEwug&Ug*F$U&CGOAbDEFc_ zf^AM7hkFwHU&ZYNA4)1PaCgG}o}Lq*yKjaL@2n{I9I}59wDCMXy-{22RFvYo0j+GN067cwqcIw(}6j8dKbg&Yntsa487E6s^}_ccSE_os*c3&igv{Sl3iYVZ+u}_fm2+C9?1w&gm)XAx=n1^2pq2 z8gqdmMj;9Jn_HAN9*oZ{( zxI4Qyn^Eg#N*>;0*>d$%@^N5uF8hnrL7IhPE@YS-vUNDCaNXl6JwW81cUo4xLt=9 z8Z6*W-r{@nT9}9b6zgn~V;77)RFZJ3k-M2mmPIc-0%=Ky{_65>@}tn6oNVyC4X$1+yH z$LOH>HtCoAgGS`Eio{c`p|o3@l82yN5o+y~G`cL&zn5H5>XAqE3WDu1@z)fB&av>RRkRQ`%@1f#)^+M>6i!Yb-kwLOUk*}=n|%Y5khl$hMq_+IQS zV-omU&^vWn|@0B#j&6Z4QA>%_E?H9r%1ov7GtPJ@j7?VeC7X(e3~^P(3efW6?* zsdEYcRJWq)s%4l{Tz}6edNV9zADDIVdO}}*5s6WlWvNL8XIfrO=5wp zJ8Z>r{WMBI)$p-dt5=}*xHmAQW8i;+Nl&k`H6R~IFT{* z?8MnPe;Ud4Z_#(zC_j=(SQ@ydt4axV@W2Nic%u1vuD% z`s2nc_6CZZM5U_qnK?$+jdJrxqD3VFN4Y3NigsU36{I2o8`4kCc?P2|DdAG7J3rxD zTf^^jGVDa@az4`NTFG1G6G17J46}D%EncKkH4S4-N0NmXR0cDzE$L_O7rG!%2IOcM z=JeD@AjdsRd10c738rM`m`IbW&1NM^A>R28?@nOcWn6fWv1j_G=nNe9QiYCDc2y6rZMPV)+@~@4lEJuKCHSOo#cv z$W?$jnd$kw&NhV*8q;nwi}H4kY6Q+^KE2f0(C_dWm@@IrxWzc95YA*wfa(<0RAf<7 zCCR_TxvO;vZ5X+xjJ#jjljylPjXXg#WNg)jV;Czv$yKWE;=NNEdxD%(?QN>0k^MQ7 zpqCu+2p4yWo}q-Xn$aX$)r?OL+glx%f>M9v2^hx1p$P+AfEB=O6Z~>(PHI?*0cZoY zF#JzljjyL)bMeWwhF#H_*u3)kkwZr*xj{~+>D2SOdgRH7^=4{kyUNTBTZ$DUPJPaD znVM_j6?1CeAXJghPLolU9Ya@~7;#Vg(e5hcFp~J==ca9H)>I0fMlCIJI#er#8e>Nm z!)T>NzMu2Mvhu}Mi4$TT!#mS+JZheI4}y;2t9|noTi+QZTfca7ED8C1)B95CTZmmM z8nP;`6YZR|dWoA(XR>z5OqjMJZLh0 zUU@iB3Qnktt7_mNzrjtyTcQFUclzI=9Ps|0GSYXhI%3Rh zOO0$_!b@xON`Ko&iH5i}K1%IH>H0YONEIk;xrdKqfTWW4u_Id_z^|Z^dFtsr_M6tBLDp zU=>5^;V{zAYA01aW2CFt3_>e zk%#n$R>M2^ore^!JeZe(0m+f7kB!U~#gF_aC}{R>KSoDasm2eR_j3-aQ*>LXWNy(IW0A4=EXfM7|a#8C7y7>Em(! zBUokD#xL}U}R;=HCUXWz;(F}X68wp|k3|0SvfXhB_c|cvnRSw~U z*Ym@l4AlL{9jpvZGAD5(Zhn?a5S=5f|^?jwE3dTO@x^FBDZDaF z7GUGjxl83pBi^Cy%rc(9tfzg!a|mn`vh(IjLL3vA;{Fp~N(5p~n4IogMB*j%2$WAc zsN4**NqkZ_y3!d(-***XnLCMaOz7O{H{I5{ByC)_oc{?gC}ydduUwk%%68Zv->E|< zywntSQcdHnrA)1e^TDgFXXPJNVJ#P}LE4(*&xe2kE|YrqFYg;uyCII`pV~QIN62D}|Kd7f=W2 zEp#wr5+^I=y7EN56q6(Z2~TRB!4^!6Z3OSSOuM)+mwpJ+&<-(G=!Zb z0D9PzSxzgZSGsA|{1if&c&-|$0XhQX|C|PYy=fhMK7~iC^nR&xp8Ket2ZE+EmkZol-ITl*dDQ{QhS1<3i2vy|&A) z``P2)w{30@R}ab3cMrG64`+SHU0l9XNaG30Gu%ZdtIh5yspVg5{Y_U#Z@|b~Dj$FU zqNYEuzL_KD?qwSuSS87aT(>LCg<4*(c2#vs`Iy z@)ZH+cX}!8?qB)h>-h9OSKML94{ltm^)V`3#Z`J|UfDy6n6^XBo|uVr-C~i}lsp>s zC!5@ip}4x zyXgz}Ri`pZ>8y)dxx8G@>RW+GvqV&2B|gf8_p3;GA_UzdcXn%y+{x469N2oZBV% zYc=D1t@=LTcAviZj9>-!mUnV3HvJ1(+|_WBM@?*XyO-a)H=*{_FMd0`cs^fM!#(pY z=+1;{*sb%E3tFliOVsYBXaCIYc3Z+H`CpR?FBhP%N2D8<7d|W_0%pom!*YFp{C>}u zd!Mu%6%+h($7fPQ?Y&)EpSNkHa?+luyV9}4T)A#v2>1Iw=g}`=dS;1FD(kx^-yh|& z%@SMc^_Eml-XofHj-}XuZb(9`Q+m%r2{}MGn7v=c!;CEqL*G6b)^SEEHNJ*V)u4m0cC4aDNw2)q-Mt z`%7$+>dc|eUHHjjmN?3R%YpZ|O#%~i(Pnp_Y{p8}Lw?t(`C(@}hPv&rn~?$JXb&-{ z=Fe~OAAex=9L$t_5doM>>K|0~JaZ^XcOBBOHCCBjch1LjrB}-0y3s%MTe`OisfLEM zYnI7Qm)SioV-h(m-Rd$O@E9lB9}d%P$Ab%;ru{x(=_P$be#F&u30}y5Z*((_|?3s3zzQv1x7|w0Z za1O>fq6n~_D%#D`yD%hCwadVa_O~R?BH0Ec&9WJ}5{;7SkfPKnMekDlyU)K{GSi3? z5&M(*;0ucl>1UB2BUhO1;Rfo<6Zk)0$I8j{LT9LSqo2``4$~*Rq%>`&(8t&wJIcL# zxCMdEg9ZTfO9?wxhM~6pZcatuvi@R*sIP{4Zx+#rNvKPY6|&n439?Yzw*DvHCVt z=SB+I&_5S)GG8!%o@bNd{7GJYwp;3-=Iu{H)~DAsl3!VH)CVPgblPV{PsA;a~#v$sZGkLD>5ZNfWo-0-p{@^3<&oo*0knpFxb{QgUQYvroD z@KR6zYJ_!2Uu|8C2@=JTAR{+DO$7R>dp=AP!yTlj*Us=v% zH#<89gRbwoB_MP@MrJ|OLh8TL5;BY8nm!7kGR&i}MA(@&W9ClBay5L)Gy0gT+NlwN zEPx{IwSSv$7@#j_pN%E3G!Gt04B|GeCjpkgM9R*oRMA+5n(UOfLF}u<_NLydO|bdE zK=Ws}U-s|VPI3y{#jFHo4Mr1xZ@pZ{AG9iaO4?E!w7yN)807vteCKBd65oD=V*&Rm(7bc|8;#&g8k;pYme|}MeX*h-KfK#YOPdn zEC8y0X>H@fIbo<^{9Oi>0{3HwKlJ@=rw@U@7vy^2n+?j(x_=}qz3z%LJNWe@(Mp^* zRr<@rx#h^|^ZjJb;ISmSvwQ~Ri}L&F&<~-uXe{=P1w0cLEk9=cmx|AJZ=O5F9e+BI zUg**j3hjv>8alb33&~tQIjm~-PD~&BHS*wj&rwMJ*7RXtdf8&{`3K8%Ep|x<>E|n7 z${RxuEC!r%TJ?w@#by z{arzaudb{t79M^p;N?l>kfa>NTi-eUGT9uPexFxve(NAPA97ZFFmkM(+z7qZqmIhO zXXBE5`8K8+V(Q02+Bu#i|KYNJb=3mT=w0yA~fWW_o zH1z}-^Vq2Q9Q$xvPa3u3z=)J@tJQbv2xY+_b9c0d_X*!K7VfSmC{G9NR_k@Rvp}Ug z)TUqN>(>sCq@7tri>&FU>+OqtRWw}810)urqwtEqAfTn^%jLJdTzgzZSmN2f$|*Yy z=?Pyi-8Z0B>pnwDBL1(PH>slxPVLaVGweDJS_@pV4^YLqeJY6-dg*FIqkBy!hSk*z z4o>?UBPuRUIp)%|h+CN6rG!|CO5;egkbX*^;D4SPZt{oEFP~?Fu|M_k%nusjb1a%v zy_<9FO!bYwIg!^WLfsxYi|tvwMx^7WCFSH3+cMlAalHeYT#89uUyT=DhO^~0(>%5{ zYO^PmXk)=_(^$}ZeVxYae)9Bwy5q6H4r4+BGNtD0%sPdF<|k|Wg2TC_f!F-xB3jU9Lwp%^hnsxIPiGEOiaJ{GL6*36@fyVP>yVsiya9}$B3s$0)a3)UI z5Y>A}#{FY(TT3xGnd&H`w}G9Q`@^dPx!{%~tMq4r$g5&cOH2=t(Ru=XIQQh}bmRw# zIg!#;3kZ;8SV5>~MGfgT!rR7phk5yB?-UNc zXPK<8JYHgzIiYyhekb!kl_SBJeu_E1Iu@82hakcfw{k@HP$1$y!&uDmusY_*f_3T& z&7t8E(stfp+lniMvU&}QnX_uoq0K)cmo;v!rHp^5eFvG;ughm5#MDQ?KGor4)2Fjs zSoq_S1#^&1i+uTj1!;9cIVZIJo#}&EWhG1gatZg!Z;z2}I<^`jxVNI}IC>Ix&`aY~ zvyNhFMW0#+BOP9*lc3^+xA}-#A3ldc4!F!|IXO#WQHDs2ZfD%AgIN+NQEu$e zD3HRRb-;*L;S}BMcG|VyV(UiU=|Y(AGHP|Ui-NNAUKlvCBFyt;ifpLB{AwQ&8}3Jx zXzzvhVr<0zvFFQJnVylDQJ{$#2*PocLK#z~Nlg9<3BImy=pUrZ=u&2|Wg!bWLU$a%z2kcI zo#l?Ww}(#UcDpyfWPw}R;}^>M6GW6Djk9dmX_z0!3;^!h&tEWn}rx%z1zG5tcZSgSWT}+YnO@G=N*-N}pc!x4Ell=aj znjGS2y|?c@nm5)G#4nwriCoTS4$i2z*z_)Oc}@{;-=`6Q6W=<=mDo{F#SC3$_)wF_ z9Q2~Su04Udmf+^11uSf13+2k zQoMm72vc0mIFnKs3xab5(PcYKWbdGQv%+74ELB?i(&39^xr3Q}(P+kZZFQ^B3*vEq za{a*Q%sNEAHrW~x3pcZ6-vDb`)pBk|7IpeNxX#IhMm`alRZ?r%XG5ftx6R)jn!jYC zROUBtmFSqzt;$m>gyuAPE;=_mRCbHnYTRS79eImcQu!{nURqu>J;v7H*gCUgdb~Ka z7Zy%YfC~cAmTxjGR#WljWY$Fi^T+lL=0v#jmUDC8<>!wLiAqYHIa#Nxja3y++3VhF>x!74cdmF^j0qS%D|>B= zQ2`US;Zt0NFN(h}#HA#DACHx8T1bo&)cQpAt*x24nbKZEUZ|L#p>5h|gAU8}l3T2c zO2VoB`YtU#sB(m{nZUvGw-_Cw=PgCpzB|=AVZ#+JK~7&kS{K=&#bIA^On^Z-ZjPnw zQ&O^&SK3vgDunD-)kzzk^>02C;A)JA;AwD%_-X*Gdm;`r<_>G<$t!E9tovV}3azg{ zVl5lO1Y!ob`u*%Zya>6qJ}ROF7AhjF2y1AD<^Lt4pUiYnh!}=-2D^Ixk@u#UT)z3& zeDIk|ow#r|q$rkis2@axAb$l^0%QmCY#Vmw1*=7kCgEmTL%S*Jie&&)*~^zit*2jR z=d(r(G#kV=8W2JT%#q#@^OFKsk=m-I#{;}z+{Xbb0NcJ_#Ai4~&)6v&G?+sp1b7BL zL7dlHLpM;lRu8j5DL8do9W~h?Pu*o)6uaz2L$B(d{(1R{7pzHf;6>;b2HrAl@&nFoE1)NzPVX{I(gOz$|{`Wtj! zpvD(L4+#VONY%mzcrP_YI9l%*cR$W(MGiD=h@r%SA=n1R4SR7y6Jg!4SS!p9wL%{F@CC*3T{yHcE-BuT zrI0*4UQt+-os`;02YhIMurD(Z#7jkzuqdAd^p zVa+UtKD)#w?YnWm_cDd=yCYc_?IOblOnEIeUdu}2(0vU@Wa}ftuG_1+A>*2l2cZw< zRG=!)ui;>m$0bFNU~`T1VF-veHmGeJv4h4-dO)adg**s7!nEX^7e*cYDf^vbSmqCI zgJj*AX$YXpNP|I|og%!33oefbon6XHaba`ktKo=NWe~3tDF2uNn0E{U^QWS`hL@bE zgEj@}!J5YV-H^jzAdv@1IAT#A6$B8M!VrrIr~tL^5o{ozrW;_hTOLMyb2tb^0+^Cv zz@ya-S)_XeYq?idM+KOcBD}yu3IgcQ0HYnla-2=Z_7Ut^JnCOvHMZ{T27}3fpdwU& zIS9A{TLM&m-ub4_;fNcUhSmJ2QYI?En*48DfHDZMF~bqj@46wgGf0E`{J8)4Qw#^Y zp<7dQ0}nVL1+pj=;oV7BM*?{J5w&E1V-PREE(9q0=zm!ByukDPFNXeyoY@VL&<{ss z+5kzgcSBC!W5OD>$$0^`Jm`fgCd~WT!xHX43i#dX2Dnj{05=NNdO{x;*2stoa5a(t z(Cc{tb`>DxvxGCT@y>UQg(G&@9uHEaqFQsQh9fS_kYRJ>C?l9NSrnR*ymmD&!V#~# zoh-W{K#1EQ8F(br;`e`Zr#jQ^X`duLH};OcLaz~wqij@)%J#PSeSfJv>Oe$#U$Ee( zLw~wKMpc+|#h6stbkkJJ=0^sQsJ6%UTIICI_zeVFCbu_cR zPZ4<67w2B83~x{e_yQ3}rOu#S^AUU2M2ebjP;H87U&NaXr6sA2H7ka;=ocz!YH~A= zZO8XiIY!dbm6j;@3`Eh1cs@0N50D*b zKr*aziGhpPxf-=p!v*3nG6S{@zjCl778j?3v&O^q)HqoZsl>h1xuMF!rS3+C6yCQY z0lkLtCAOv2-!n^;AgWDpfq4#zt+1FN#%Xj_M}w)p=rWe-ucmqwakC{I(@ch6xj7QK zdm09&wZ(9jfNzd%JYR|mJ1Yq3tB7#m;4oi4E+=No?VqW;d4q2VL5gPm1^s$M}m=+Dfh^di9Q0YK|?69pG)-Cl6AyGw2I8R{iSD#^2g@G zA*|=jr71OR;l73KjHLL>na-4ZL!2gh)A&3FK2xkXT6CTsv+xIdL(=C+gx^tyIV-K< ztd(;~cT{XPS}_pE*%E=9a3nH;gl|gqF&;Cl3b^zN(#A{k0RQ6cW5=(rM5lx`=1T-x ztPTrVwr|kwGz}@@YJEN0kygNY8yAf!53|R4&fit_)>MkLzA*an+A@`?4Zvx>W> zxQ6SUcvYXjRGbP~u|SBz!H# zEBrn;2{@oLKF4N0f*W=04nH zs4E63WlgVuKL$a_CtV|3N_gG?vo5Ujb8AItlEi8;QhZLmxEw~_tHy=$z6vX;Lhho9 zm)^lyWa@fyXkGFu-MSNdFKqDJbtcR2wyiFYvaSw09l z(7F@YzYNwY#Vt6YYRjG7XvOK2CKe7Y+%-nhQ1BoICnO6g+)cu#X33Rz0V&nyXJMs9F@D6DF-7{RC9>OUq3$0vs|eTTU8UeYqV_0!-h zOn)XRTDC_5N6siSqC3Xj?9JBO5=B)Ldr1G6=C1roH6}5%F^EvN#6ngzCZ?)qjQD4+ zH_CDtMlgE`(A5Np-q_SLkqVM|)(YNAL=R*Xl8mynL*(*hw%vy2p>IJ~jiosI>C{+7si=rg*GkqRAZG$)Pb z%CP%4q*uX^n3buEG}G1a3v&-qG1K+h7-d5;^L-CdvD-)SQuqPwQb+OG<$IpH{UhPo zes^7B^7=V-cR4Ne4sWOyX+UKCW(<2#j!-KGJ+RSjY*&Qq+P6K%vyS$sRQ()VFTO&rFGIHUb$r4HX4Ekhjl)us762 z3#jWQ&X5@NNbq3YH@q-4lr+>Zt?O4&?S&hD4jJrl0qu(?<11wB`gT3RI0+NCAK~S& zXdX&}2u;?U=~mg;%-5WEy`M4|?Hb`&Z4_Ptiq2uEg|K9dWTL3w%|5oxRJI!tom9>G-;|K(vdd*1SpK%vO%a z;ig01E42*=7lVOPt$5|)58b;21~01i35BVg>Kor(`%dIsrNKy@)=5>Zlmb>pS?r&5=t7^`Ft z?!L2($i6HREs=^>mNLU6aj>axv}f*=)RRKJ49lC;mR3cMnX}Ftpx9aN-aYLnOI`=s zws+Z;XsLLOR2E#{CLXG{-Y#^j-9)_0Z9-Z@dOX}T$Ri5 zsuIzdNMcUD&j4ehPFT}-K)e5ZZU82cx1<3ju4WNJFV!t0iTW224L(wUcS(rZIlYP( zner`@9UzbeCrV$RQk7jR+2xA>tWfnVXj=h>(AmOT@V6P=@vrAt1!CW#`v93S0*<-J zR4HJXhtbm?MKaOT3G3%qn=1~5UZH3u*>@!JTK!_W7bP}{T!Dlm}L;MPci0d7gI=i;&On8{0U-;Vw)+r@p#Q!LgC!AsSu>>TT?C5 zInop+LX(mzp?dDQ@F~>}A7%K>jOyM9%E3dk2Mv|dO zSUz7K>!xW}G|w!9-c{z-i8zhBqAKtieM*CH>Ir$~gVOqCxRzPVnZAIRcqP&!I zpJj0=hN#_O(pqQ{_&zQo30XN}NoK(dr+-g>h9mnbz~Y^o`Un zChhA;0gk1BjLAMqvZyo(NbR9~lk{&>Fmqf-Md+0-o)R|o`pG1~ zGD)}-H>UpWQa5y>&##97+^sT${#|~VVef^FHehLPG$#QsuyJ?a+ThliN=c#0KjAO* z?MB5oI@S~Ae7>QRGoqw8+_VSuYvhUBSR48=NyT@&0XpIK?05U4U(GGNFx=`RpJbIKz(I(zOfcT;Z99#M9g9u zI+7zkf+JGI70-q4-iyK&Pkr(HfXWrsKLb#bM^gv-cU~e0y=0=%_!Gn@;auaX)Ok_B z{LHdCB~sJKZ_q0JCUR0Jt1JMBwB2;yP^C~OFv`Bt?cCx*ecalkbuxmfBhr@)j;`pQ zPU7QdzN$Na!2pOH?C=v}6AtvbmC~V|amnySLyNiMUx%W5=wH-DARGxF;zwh}er8ow zmNx)^N1dC?d_NE|luDg~!)kAI2f&D-g-QoYXu>26ek@ut#G?QF@y0t;~^Q_k->P@C}ZgF_8tot*G6$fT)EYz7p1F@9koAVs7l|u zPmXB7LzcfrB)ZKtGse7<&@4#;)1MY;nnTOB5~D|4S*rw6Cz0WdBGj33fPfhBs zXeFm?ko{iBsWAp{>sX#dOgPr&trG4FIQo7Wx^19hNo=2rpDEiCeliyHXU0T<2;qxy zd_ZtfFV{v`^G0%dYZ@Ru9f6>WbHxZiKOr$8`YQz#a!-H!>DCwW+!zVtL@;`&!n zv7%B`DfiErxA_)RS3c@lC?J4NwK$k*#F<`Vy5J<@4<#gQY;!e&aWE8TSZ%;r;l<6U zb7qvQC#R>bOAbDueFFMX`qlo*oib~?2Qrg zxb&fC1}MH9u}P9FR9ZJRf~a~$kVFGQVy#o-dk7PHRgC?C=>44d%L^86 z5Rd!b3+mGq+?FQK2kDijr^9s5rxr z%Y7fkZmPt)ixNtl#pZmeG3{o;E}`^EuX63O<1fcsOAWi8sinuBP9A5d+L|j5Yqwr2 zyG`Wj?rc+N`CHussD|Lm>;+!&@hyT*$CEl|fPJ~jtR7E`_FJ@89!x2t z4}In0XKEF;cH(-{iOzOZ6P-~u$ziBun~zQ5y>uC#$Z4XW>L*<{6*{n!xdk`1cLbx* z%B}126xwb?37$x3PKP~ol?jL-ojr}ecAz>QSS{Pr)1)h@MG(q#6~!UUi8r1b8EDWW7zRgV+!pg_S z69k@`7x-ORR9<2O3Afp@*~$KW6r!pFr~~e2&_rPiLbI431e-?E$C**am6J$BP-j>@ z@lHo7G|E1~YHZ$KuWo5qYLHGvRFfQ0-j+i{9Eyg9=c?H8uL&;@vdHMyq_@Nz3Rjm| zjgrR0Qo|vd)ys0t?I-r9n~vp}=NHUX+u`hBu}-1P4LJA$QPNZ0)>VIu&9sU*xYR}d zR&Ppet^OvMhOW$_PYZ*{jr9{5fhy%i4eL!_1@bIxms0r!TvHV1{h_@Fzq*0&6QGO8 zs`ffmGv6I)$W7ielK75_qz6+~cm-#MrR7>ozv)Ju@JUk@b$PEWo~8D5U)Y(!AJm#G zz(5tgP9&nU1VPWjj8?IS{_vh-U+qpfxuMc}Ool7D%u8IVHMz_#8N})49jB}sEmuCw zr65GI>mB{9S;hEfdhSkq=}v|3_;xfH`$`>K0TgmFgDWtHHRGa{<<;Q&4H3qBzR|BQ zS@x|{E5XOUU`-Y3E_jBOoX45OGQ9$he_wEn`|-ZP5KGkkM>>hch|vysIhqNljo?C>3A>>Btv8E060G)Ew~_<-3SV zI!72n>GfJN2oQN13k9l#wbGoPVhibVq)&ZI#@rc)l>}8VYG!5&WX{*yG>_Xw;!|M7 z+CI%}WaFl#pYN9H z^4Kw91a@0;*9T<#!`qSuFole`h))DshvB_lcof=SY1nEhIEic*@nUiBdX-MoY2|0z zI}<(!uJg+&AF>-HpjTQCNWe!#i<<@0FWgRz^?xc~vJFa2;spj~1LE0x+1t?hLWC^^ zjqIp1mzj_Su1mp2j6*Rc7yEwIF#8hVh!izo#2-k!NRrk!PI)&@K|fSbXr3FPp@xB) zmn@avZ#;o=R^crEN{#!>rTT-i7C;TF@1O@@mwjWQt0ygwr(|GjXJTaupl+pR#?yGi z=U`&44`5RjrekK(Aj6Y&(6zO9#RHJYey7oOwg=GE@&jmk?E!BQ0I>Jsy%Auir=_Q( zqorr11^l60J0O#kkaS(e6GI)BeP{t6=M3@caT4&GY-ev)0n^KL6vV;McX(*R|DsX5_WB7cj8Xvo*1{ zx3UHB`~nUNAemLLQnWOArXj|o|DASL$i&vpp5Iv47LS1ez!j_e`x8cbC>muGeS2d& z0JSa?BOBld^mq(Rw0Mln^mvS{%y>);40w!eba+gF?Tjo8&#!Afa~uE89sEpK`>%lz z`~iZB&&pi?S0v9D@Nbd)PL%uKAz}L&34lhJft3kx)@*=Zrtb*ozoTQtV_*g#Vg(=t zu<$YgkTL=2e4if~0qnaBObp+D{|n0RguwqAOyPe3^LNJIzY2`uci!Ls4kkc)e-Rin z6Dt79GfaS>0Mx_Z1;xgU$IQlp$IQ%x$IJ+LjUE7#h4E*heJ3^k*Wif$1&*k_uDOZc zFLGn}ovQdJK)(|f{~16G-?a?@1R#fJd9W}718{nsGL{sq#1x1fHp z5P&uDcR?}z1nS3vqNfE|2wKMP3+l(>`ZEjbXGi;+llqk#{yPi%@1Kds3`HYiU}9tp z;C2V_l>>lhp&L z%zqH>191HVqJ3aJomPQu2T@}Ec4(ax1U>>d61;Kltt{~pLcDaU?N^?1ojU;;c)f>(y1b$GlQ1PS${A>cqtK1h;(0P)vQ ze{wl~A9e5_f%v-z{Cg1pUCZ(N42S;+$ls0Z-vjw4<@ir7$M2&l{v!~7ckzD@;=gM- z{v63j;Q=CZ-sd?mme1~vM!>+)M9)A@nC~wRjGmbe5F!2}yO0{^q#!?s5-`%l(lm-d zfJc0e;R=a2&LM&fCL1XFUIxV9m&8%vg%A=vnUcOjE3j|?fu0}|^ar1JGmxinGTm*m z0)kshA!d-rB}`|_Ne?}$D|0)wGjm4A9y9eoGWR}U?e;3b?-?^xkdF4-G!PI6I=t*Kb&$Y>^?cn||_&Bw*nu0tbQ;oML|K zNrJis5+h&Lg--tj)_)#LM({~nSr4?jNZtY7SbTI zoJ72q6#|x_kwCm^l;sX#(4-)m1hjSaW@6$xMFNNxDN%cHha|&1kX{f8{V4Dh1n6Z< zCtIWK>);zU<1`@WT%*k(ut3pwx$rXyCR<;74R24XJb2Cbkk4FBXFoR!)^PJEgqyDAlhucpOqj-$q55OM2@ChIjy02z|YTnP4 zb6~v`ZXZ|`a7IpnKy6)E6Z?X~^}4nlbq(_y5T&1Xd-O}5-j|?bfG%3h69K*m6(V4G ze6vV`Y>)!Mq=?tqn028dkv5y4-~k!B31}$Y=oQvGR*HJHzuK=CZJ>bB4-P4M`j%Drn1xCys*4*R>E<+TSy}y)xgYrVDaPdh|Gx; z;4oh9ynOJ5Z)cSC&X=|)@P~8h)Obf$5jM-)D2hcQ6^$h7PlPH*M3z|XtsrVjph?0> z(u5z~$F47Ii>At>C0$CCj;Bow8^tXX^M(XTS`15&Z%&F$td7Tp$Av#f?p40!q)K5z zeX5#tlYobq2aRwvL;P9%Mqf#EQKVVye!Ls8bj(b=Q@l2HI;l3km)K+O{TC3~Hl8+l z@2qmA4578^c1}N0!KBRCcOTy+y$gKzVV$WJE|5zx`1c5e4IE3AA@fq$7TFW zwPZE>yumyzwlGWf-9+?m*E#OjrgKOGVgvQ9H(OX+7+Y2Sa^UT>I@b(x=s6((YPpc` zGsf+@k&TfB<}wqlr$(~a8*)o>_gI2h!&us8%In%o- z>Y~t*yb-6gkyLHQr_K9-z4H1%*DtrG7aeXG2jNMG=NtQ`{Nn1%f zj6K!HYS5p-)V9=iK5Nm&CLRqc7!p?nvt^W&9_CSvab+(RahB-iSE(3fNmUIaYDQ73 zv+&kx)XF>YNsH5{&}f&*9%pdLrkA8=w+OeWd-b=D;?3&>^Cq|DX9Q)m=%(prt?3aK zMiz~cXQ^dzbhGN!*aV$$ZK#u!2d0WfiH7z`%Fj+NE9lgZRFA-zZ5xv?7BMySWR@?g zsul}Pe`4!++v}d?l6*q}%^9o`oQkr9qD^^7DNnUeiK|Mc`a;E`dawMpW4<%VOjFOJ z$KP~AuWC4VY5Q|Y0o96Q%b^EHl4ki5N`q7bfcNB*;Ev>u4uUhlIiTtF)vUean+9Kx zDfs=bYhQ*>LN*{s-AF4YMDtcsRfoxjUkztL!=Qdcouopu9kW^cf@K>?txp}I?QESf zskjikp|*cJC^-|lXFC@96gyk_ko=`PL4Nnv$`YNqO+&76ByMyK>_~G z29faI@m}HJP!w?#C!s#!F2N~bNkKQkctP=a6-w2rO`#CA5WHMY6kJSXHzG&df~)Z* z@qmqkb*POaY6gwCs)Kpk&szlpBc&1AZK^ipIGDKDD{xPNi0uVi5~b(bbxvDgGtmtJ zcTqdt6g#0On%4&Rkznz@mA+#F@tvB1dZMMG$f6q{(!8Hn9V)d?y*CSDB4V=07z)M; zhD>~`ud65ch;@U?DKr^4Nt-$Znjua6ydB!3o-q@)cHY9{SN?eG95G50T zP=&5Ctz0$|Hc0Q69faRL-%Q%H9Kh~7>$~%0GGuacFlu&u^uKRoIx>4*;x}EYu}~-F zq}f)#cG0{aZh>bZGWTsjXdu)8bHy@fc~bV1Y{=7mu=;EBE^}n_ z!zIJ264Qlv0(a!FQTr>Z9?CaMgO|RCO*xl2cTTI0FP#>bcI(en65pw6 z&5f3>YL;vbo@SobHYgu7D**-+=)jtU>xWm<=T|m~m6-}pS`= z^t6XNOmR%ku;jDEFt>+BbysZI^jJNtUhumi7VXL~WYt}RvEs9xtoK~w9;d;)opTBD zDBLA|NWZKZRSQzfm|mO?tjT6Jag%Uc*?m8eiam@?#hv`*ejaj}m{iqoBt8;3jGU6p zmhR+v%Gd385*&ycz^-i1RbRb1o?2R7x;)?=U#4|ZXIrq)ba$^9Uu0Curpe)v>d|!Q z{}4Kj(~uUYec)-b`}x?SY1g{yA#S52toen<&aLJh&GKcv$5z-=Dkf+!`0WvOv)hx_ zBImBh^1~QJ0jw7HpeyTx@q^yMMkXngG*j=$$#HI8&~i~T_oeqk#+}}IPvq`_^tE(sdMLM(JLLVT zGl_-a_R5v-+1(%9C;o9h>aT?YKo|-r9DJYL_iIVPT-OLNTkyACew)mPN5#m@ibqAy z3K&sG2gozCJr@CJeoI*KS{j)f;L!qNUpu|$gbfQl;D+M)lb)YY(X#-a@#|Xu$O$n5 zjw^3(V4?K<4haL@=V5_>k#>N=fGWT5!pI7Uqa}5nzbCl?AIJRrnE>Ku13>ckw`|y7 zQ@`Kq8$a$devH6l{1Z2BnlikW9e8k!XF;5~0lf+5a*+6nfdn@Z)rAWXIx6ZSOn4!% zj~=K|xpJ^}38|?D3EQ)&?8o$s$7BgO3?ah{zTzs;ylB?8X`exg8E%I61_#+I{Rbav z^iw#v!6Tx88FBShZRd5aOgr978(hXwlI)PqdSKtN@6MpL*a)S2Gvu_ci|kOdYh8ql zD(p8`Wj8b)_FdGT)Cq!#34~pu^nnG7Pl88ZBGUKOMeS6*`+CT|dt->!eV1z-OSyXA ziX6=cM{G@9J__=vi?oZqxYDNy2cF@8JUOhrlEeMvI??AT8$bo2`^DcS5rwaY>DbBB$^8TtRE1S&T z21S@bIYj8D!o@Nrl)wu%+LA1&jUCZAjhZ+5>#n(`QcSxFwpHBl`Rom)@E6T(5xot6 z(KoS>kzRQ8J>?$*NEt59dyTBxWXg{yahk&{*iC4by(UDcp%SM)(=x8s#}BtS5(d5% zR1W1`>NfD3)b80?7u6vk3fdQOrDwd8lK;AxNKb4~B9+HZX!IBxG{iIpTRUBuk4(5F zHJwN<{^osLF*1@!9k$Ah43Om1D66ErUPJdkownfS{x_GasfonPM+_ ztw^UiC0qm7@TPwJ1Eu>lSNFGt%wO!&FJ-j<(MtDQGW)Me_r0d~W0CzZQmlW*NJS1k zE1mDLD-`wwfB6B431~M!>$uJX>OFu|)gXAOpjFwf4>^jm$|C7G2}@ zPjE;lWTQTuF^|7Z&BC9X=yQ=(8fT9#jWIKv~F!{y^JTk0OVH`*Dzr+V(;} zFYV~dB?^_`jk13}op$!D-cbHnqc6w*Cjd2s{jzPDg9$R?yg z{3OhUhTHB^@1dFi0^55F{8SW*_^V!hFFyXb&VS07{XsqRyJj&1>JHB;`4|63OaJWu z0OML|S?K?x_KXhT4xi8DH~$By!vC#3WB9rB`9rl>{**oYp<2%lP-l+O5#+C`rT$md zk{d^IP3Xv5QZ-bbtMITafsARoQ+2>HW>j+1+74Eh08I}?cHy1Rx6V4x&0P+J*|yXK zkuJeDDm3Gb@o23CYZ>1b9RcO;tYs;i!*t6XnyC<{S?a2@N}C0b$}|x<`)EVcbIiTPcH4}%JBb-qyGPiqZt0_;Qf~l zivQ6{_gfY6C#Cy?;GLcgaP#<&iYl68Bc`2raNyq>q}M7qDe2~hYW(2dsH50^#Z9^d z4v|91pnJE-zudesdx+wC{hGT69y^%-YSIM|N1+%^$VX)X;wbv`EPS@n-{UCielx>y zUL@nA8RD_Y8=E*u!feXrvC8Z({mH@{u=WFHWn*&ok3?2nupU~wBlzxP0$ifR^Ga$M zHj7oSXV|yWSvZwKJ8oe^k9gr3w?d#RjO|&E8DraH$w#KLNL7AKe zyTf@-E!Lt#wy;!vOab}ho6ih`R3ASTCUC$3GgX1Zn=dn5;rW<$HG;w`2yb(KeBbE~ z1aI9li3bt~m^L=(nw);(rtJwY{m4#c7Y5JYzY@LEV`r0A%8j<&YS()M-ZUEV2laG@ ze>z70gI1RR*|F4b72v;Q{ zXyqyQ3Sthkh;w9OiXsjhwmYaX;yIR5kk{K60Jp0e^zNCzjHVRFH@JK~fOSuxOv>qS z4`)`URmI{MQ8Z!~O+GYLM}sp5E}B`Gy^XVn=IfLPW7{{dHg@_xh7Z|+S z>m?-@y*>9i$ioZ*g;-Wmi#k@P=;`xma@M|8mY`OwtU!Y%aepQdpTHe|=PNh@@;B>* za0K&TKdR&Tt^uMe>-2AECqTZdl0{)WAB~gCU+{g+U3;A`$C3fght_TQcH?UKIy2#( zlg!})2S4Xn^rlNAWa|#LzOU!;XFM(p$YK`v_=7~)55U*0i>UU@&6@|ehq>D znYNn$ky83O)cs!~rC$@i|JO_D=K%A6iIjc~*Z!}U($5ju{}L(v9Qgbe1<>KqF+694 zevb+0Xj%V6%$X|hA)&5_#C^3ia|Lx+Q_~1xUc#epoHrObT6U805;SfI2oco?O_lz;O z9S!G$0``%21!ip)sx7&QmSEktofhHklj=9VSN82d_X5>Kv5jKXQ5ibn)xrkt9t0{n zEIxZfx2=^NOE|R)Th*#mKE_TB*nN?eL?3mZM8DC-)i4C*my0CQj2|pMI8-gLA7~j7hQ_W|vc`{ygWDOP|BK zU(mQd%wxg_;<&n?lN~CsiiO9E@C{9QRb*T#B00{*T3fCAraIHzX>Scw#fhFxeiZCY|P-G1(-V0wR%Bct+vEk5t7O9&$9G?^9u(5Ux z+_+;*;ty-fDcnu}K!nk4JyNAg@Ej}*mwe+3OW?cn|L`tnewt+)3zt&uV4<+v?k;^8&jE zY=NBP1cSR5T$kd^i~Xi^a&h-C@O{31oy-Tnr!qxV|--;c67O@IWBc;bj)(qaJA^xvck6GG+ejR{VH4;0d$TJ4Zmo=emh#F z1YpbjZkCaJCY-9&CHU~WI0(Q}rujliIphejN!Evua)ajj!0MvgqZIn!4N~^{3*y`) z!Ggk+2|O$aev9Fi!3ljt^yr7W#*usqQ4(_BLIm_fvf-N;5u<5A7Tn$EdmA>Hvw6{! zTA9Ckn@x%sO@QCN^+-(1d1{xp@tnn<3;alK6MdW^S2cijtmQ?E1OtALvs9A5tTP&K zR;!AvC~IEm#W%Wy<7iy$iyWrD3S3W1IC3HK#;b1=x%;q4nKKrV%v!;1JOwA5VTh39 zSSu&&m0tzL0T74uxF|FJ zTp{TvihPRI78VZO2#loiR>KQd5LMVcZ0sIS$GMNhcLzQe+@21Id+IoM4UCe@g2hD8=;d-thf|76 z@1|91-d$!;2U9xBFa<2dW5(nzq3^vWRS$^Pj9}|iamhZma^E!CcO0Y}DmC_s663!& z_D1dh1d7p`rMZ#*_z@QNb90k#%l%%6=OhZRg~YO$IojBhvoNSUwmt|Oqx?b-z%dlDC;ZLx3_<pw~&u?;iAz{g>l?6G8Hi1A`H#-5f4P($fT)B zOkdY)B@|B;iWX`L255`@T)Aun0u#TrxE(Xdt4N5xagWo;}z|j-tEBi`6!XHG9i3K z<~Cx$ghD2gneAob`+f%rYJDT%%8rOs1*A!V@$Lf1ISU`m?$cnDgVfS*@OGAi#j*DU z{jOz3Z-+34`G-%`cTz42I9~Lp%o=R#q<8Q0+SN7T>tY}rw7DW~9m9?G27uyjufBjt zUMiuI%xRh9DBGO8vr?$XoUe!WtmJ5f8g!RA9!aEcqD|qpljXl_8xRMD_WM|APTz zJ$T=7{$LrWH5$ zaX=goXQ-TZ`)1%JR7>vMV6O77tCs8TX{}~GYKu3oI51i0I2Cb^seAKW#9!p1)~|w4 zA#-EPY{6;Jp5};FpHoe!zjLB{qCSjiXbqM&v7m9q!m@@@^PWa=7&P0AKgor-$6a56P9~t&D>aTZIqTo2=x2%9 z@tv7cTAgkI7C8Y8`2#ohTowPXZjA1GPyCO#nx6i-dE*}>+fo(G%oP;j((1yS8kU-t zgs*sY3)~v)Q zHRml!Y1lORZF)Mz;}wsZtaf}pb}jx8z1j#gn^(eC_3dV9cS=*-7Q-u!n#~7nb7?6O z47=;mo*FAR0#%mg7243PbDIZ13qAi01|q+9d8>Iur8vl}%FN6u|IAj_)`-8vBKHKN(HDJxU>OEJ<`2;p2wOsdM8xpum%Tc;b1rk3aN!;Af>h9_T?)+PlVCRsl z82;dMtm=5>juM-VI5YCU?IY_ z_brp3z_P|<{1&zt-++G*hunejGU0g}w~R+U!fM?NN0XS<1XrYCp=IRaHPHRQ5KnnI z36bjz3Va;%{s0?2rYQ?LyCh54dQ0A|#+72nRB?Z%m|}6|dbx%Tvv9kM1*^}Jiee+W zHe@OO_F6$Hp`WzUTS;xZe4$fDkZ}_f2KrIX4|c;QfE-H>7dtnsBSvZJ*wCF5UAvo@ zanZ|7NRqd$-H$Wg?=?pDXOt8_D?QLmZ?)ZcTp7C57Phz9nTcnU?M<{q4Q-GzxyJ0F zxe{j$n#gp@t;grro%njUUkZbylXvGIe2!Xbgl%eRcFpsjEjNHtRAFr+v=#G22SY3>3RXk9b3fp8h&Ux&6=r05qWd+Z~wolx<( zrb(3tbdQ)voE_YCb}+Wc3BRPgE)CX@lw=eL-+jZJ2_32;?qpSsEp~y$8H3RLNqn*u zh8V5AL&By8lNf`|%TGvl44pMxj%j8ESj*5_OwD9eHAXUX83Z{BYx?}Pb%47Z(`Wc0 z$E00YMGKk+4DJpl5)6&{agAbKxdSX5Bq>=are<}{et#>gmGU$k6kDvrLAt;@97f4i zE@g8>BCZB`$2#nFFnNxLMzax!Sa`%G*{KcUm9p+&M!A_=YZzu~h8_@BTo$iDMj^tR zcNqEmeTGHx^zA-&e(9WR!oa#$vt{FcCzkwemK)i-uX-@9Y-1F6eaI%HpGH?mQZkuw zZ>s|re5FPMB{;#aXEeK(mZUW8X&m5}t|^=I^^(}sheAxI>|Rh)!N7+ae9W1YprgKY zO$k8Zpr1|}+g7wOSvlR>1+}B-a#%Qb_LT3m45R6H)Sy(`{3sx{m5_@j(6&_w*ARQ& zFufVd9Z)cUS4y?GJmuw}vnmH&+Nx|g|F$V!eOK5tb?D%2gjsG&Tx%KJ2GwNGIC?O` zf*B$QB14fQ<7DN&w($%1KeCTMCy#%G`2f-iz;}jz*hdDsKVcsY6kbWlDOEQ;wXAO<Vxi zyp|98i65L1!lYf+Cxli>FI*Dau8E<=a(Lg%Q|t&WXv*JcnA&wT<27}x)){y}Ox3*Q z8kK+ylKTeu-4|U0G+0wBhyazs8q7-%#6(O8P)?tSh`=~wcD{}zm1^H*A%NDR*EJXu zWAh_;a3EzO_X-Kga-zS&{$Rms%M7|Mi*T)!EsTZ0lltc4SG)!sg>eamawH4k^iQ~S z;X50lUtq)6A&BBO+lQ^2&R-gNUw~9*0KRjrD!E28w-w2WQ#)W%d%{=Q6P<`3IM&#D zmbn_l>X;mj1F1Al&=rCjvF>ZW55=~F?op?!Ea`A}kcZMW$+1Btl}EhXr;78~u8Ol| zvWJ7+NgP<>5SOB1#@lszN;(h9F>R=fGDuVU7ENomCLCp6|fUxa|pwCfD zaC10>IXE#*6bU1yif_Nijiyb#4DesWxR0(AfDDXue=p+5(NXtqG6jR%tqu=&N9(btM0>FL z0q1Eot?NhwLu$-$?!ktHK1UMb?8te-Zeh0pa0c?~3Ch3ITI?9&6Kg_xoowMK~zXn!adrcy1!G*Oo zts?G|%2w+G^r}r*}k`Xx)4Ef=GDw(8iiYL{3 z&0i-39?N^>B?CJZxJhVBfcxWS>?K4!p7 zaU*UFHM3uR!a%IO#|^DBD-K#&j{HQWf5mvHEPXg>Vl&SXV2{N->V3LuEJbgVelLPv z(U9L$L?b^x9v_*T!kEK8B{;3bLrr%6J}q&OSj^D)-RHcQp9*bQNsY}NCm0UO6#blN zaBWXiQ3J}ER+~Ld_9aRh!e3`swi#VanV&IgVhxk44{}M54s~sn8K-y0%3o9*T;03O z!a-mozYE^A>?QF-Mp+%2-GXrzjrJQ`8Z@G!DQ)hp^|-PA7LnG0fNd-vqS!W$s?{>; z`rch7!%OkTxh_}%z=!Od_>tZCNiS58PV}|efg>p-y+dbwIY^4LszPH0j(b383X+M} zmF>4$xPbi`cnl&Zg-%y2f8m}qCXw~x`!`)Y^>Sgfe&Gu)eO@JfN%xhZ>&Hi zC6rfjFX)Q(?3LGfjF$uHGAiqrQQOPjEqY!bj`y5_e$*>$YKG%p6u{EnD4~YkE^;Kl zUK2#{eNrwOHNiu_2EvD6Kj5zO8&|$~%jxXcQ!&G0^wyVZk7wO@c<)YZr^jRBW?%IY z{vzs{uAO!jWa5)`KFLG$rm4!BD++}i%|n2q==i7>P#6TR5{Gt}4Z`VHgEv{w5p6p< zUg&`NjjLH35dk+uou6I+zk^T4$HK;UwnYy6_*Dii4D4zJS}Wg=8qHTbFfKcirH_ps ztC8?fG0{5pCCaRKwrdra%%N0m&!Homg;Fry{F0@xSPUg(CSGqQ16VN%=AkA{M#BU6 z>DzeRWK++%=*y^Fl?AO^wSlTSwy{{|tUf~qbO5DU)&lv28i{trv>?<*m}0^>{oou# za1f!IJ86-ecxCzk%`Q|n3Ca2^cg&FLXpwVZ1Y+P?n-?`ihkh=*s&|e#Mroon)Yf^} zvmfdl=>;rp>iX4*GW+>0P39hHsjmbC7=i*NoW5#$V$ut4G|r3{!@XQSm$kf?#F z2AyVq*$i)U@t&Z?nu^;YO%H6)CVEle&Pk zj0oJ8vl`w_dgRN8#0O9P`05@OmX60pAqi}3i}>S<#Zd>552U5#b0|!<2Ia15ei~n0 zGQr5*_vZo52jmM(t9ohcAp?>@VmMW6q; zI3B&jLfB-?0B!qL8dLQZCM|qmHa&A=GzBurTxKMz=y*>0Y^-GHZQs$zLa*}*Y3}iz z5D1R+1Vo|4=m)Y*x~~p_V{e7AgPJo>eFZgRKtnyh&798+j)YpiD|!+zTTXbgT_5)g zmqK=8!Qz%2_Uuv;g&7AqQ+I^A5JUeu%BFFsZ1wj-fbfDSdt;y;Lh754dOE}<&oR}t2 z8dtOaMu%lD;8{tb_&+d6KUb)K7^MH2@%b_i0LJIIm!QN?7X9z*tS2eQltndO%MpNvI-S z5DOqkksdlhx*#A(?*u`bilHfjR1xXYrFQ}8Rq0%iF22dV@7^cm{nq=<%0Kg6d*+-u zbJjV@&fcF*F3)b;8=PO{!j@_q6K^T2ZfvC!eCB}|N{x|IrTfjBzHJ}w8|uM)_8;b! zyVZKCJjRAyS01jE|K;(;^)zI@4f3%8g+(EMuy80ny2=3Fv0l*0VuHm|Md$WuXln4A z!Xm8mD7lFq-(ABSKQm^PA5y&QD*a)7@bjSKvd6q&;-$6VsKRqs=HD{z&fn0>9Py-J zocy%_9v3p@msxpmi!gCebt9-^KL*Df`ytw)C%J6;xUAmVbKfpoZP zeJ>|nm4l+#wx2(8nG{OCK7_Oz2Pt>urqFQFimOtU%Tkryph>4Q^)HH(OFOAVj((GV z&N4bnS;MNqyd#Pc%NIIu1r~W9R5(CK-~$$sGxsj=8-Kc38hv@1_o;rJYDUzXZw#`L z%mlnC-dBt<8XBVz)~%{qL(NIMW_76wZzqw-=0!-beCvHL@t(l-*bs*i_K~*HB!b}zlVEA2$pA= znl(GNaV`9Dy><)?UW#~&H8-*G&wsnfQ+D+qZ?A>mxHm+60`T^3{O0W~e*+Tb#EG8( zyuCd?x@kAtJ*wDmDKZ}n9q>a`U2~Ur4}4+(Z*QooIBLCmCG!_(ZdV(7nz$c~*(niM7iubl2g_bX$HgIeeeNP07aDVne^Q zq;{YD5T8q!z3U>Ol5XUcT}?aFNgY1k^=>_~szov2>!?n%fv0ExF`dv9BtMQ1)%wkw zkb1nT{@KFnPKCuCb!6`N2v**Mu`E6D-2H_r1h5P=-rDwI!e23_bvYI?%Ebdfe*=9NG z=8obO@A}epp}Bq5rY`}bRS)J%)aLfCXP}llFbTBFg>$?sg$7y}twq7oA$HV}LuAHq?9pL;5~_6=WP7RmCGj87F4Gu6#XDY31Sg)Lkr7PXxQs*yG&EvfP*w(Oro24>XG{uX#Y(XVEqR4N_4J(9w26{Y< zkt>#aHI%)mA_Y{X7}FMB7ZECa9RHrHVg%R1R84QE-THFkNz~SYxNQ4ShU7}R)^>4d z$%TBQ&y8TCNDR4woa-gG3#V~U+{AqGk-sKt;rooO2ll7CceKqtZ3E+1chFU*Y)ntU zw%NEx;Py6`)Z~OGct3X>Y{bqNh{>hJ9H`xWRv$1(ef_}WtotmR=7DP{caXBKYxXa7 zsZz6o@?!qxg=IF=8f;1H^RatR>ffj11ghN0LTLx@`7J_BFR@S{Zx=nVy5^EJ9+1CJ zkl({zdDNTC0k=+<4k@VQVDDIOpp9~z+aq;vAXJ7I}VC$3SXA|Stgn_9KIw;adU|}mlDq1ebOPLP3Y8I$Nk}f z2bb43WI;Wiqt%Oz(FynR&kTvsnY=U)Qd82n*_1lun zo=K8o2Z2kd-nb^Jmu5COMxDj!8>&z#6dYG!>8@U`UsI)4Hr=s?U}ax5hGtUA(#x}O zh%a3kF5C3V-jp{m+pxg#dOvLR<(Ej!?P1=&yEobHpR|oUKd~fncS~C798;sc(njFP zo{7B9cLSwoJvF;ham;;kTm8eC2f^N}KAZL8AulLTat5MOoaLX>CT_Dxd_VpxaC@uf zRZw%hh{9SJ1q)p)3x}XX&BDT!wZ5ujMza(m|78pNJtFDvEet9O`|VFlI0DE4{a@bR z98C{Zb#vxl?VfjrRCFyu!)S)@>O@oN?6TDD=I_9?SCS#uSPpb z^ghQ2t**tKp}Pm$KdB3r`aTVzr51R_t*!G)jUQF@Dy>g{$~7F`uku2o{b%63lub~k zoL@Iy<EnYejai=kvx;)(c(p*!1~q)&yLx5op}tz+#j>9VCnDRdqD_?xnnz0}oJzK5^PTr=lwhrs z%`lq2SRca7E;7Ki`OC_4F`S)My5PvN!(fj-!86UED6*Xu6?xt-J)sQsH5B_AWUZC0 zJfvHOQ<*5mCzvhJHxy&d)fe68(_UMomZn8X^tHTA*8Bl`X=PNslgxTQXfombw|kLV z6IT}fL^>VodUa0RUrbzKtxURyE^!a-D!+r;TF4xK7#~Ph;p%Yq59N{VIdDrD58@RORwDv zxiDmx7@?prR$ZgU9Ev2rl)-KM*lxlvq!53~X`{kpa9Pd6wqir{QjE~cA=##;! zX8F){r*j`V6}-~!IJ^6N3?9W#_Qq?f(5ML8?u#feR3t^T=<-h$}TLPTr&LPM3=Fb*>HzR=E28 zsln;&IMe2I?GR@-2JQ*o*A#*mtNF{lbG~0K+3WOyY_Z?3yJFn*_=E*d$R_x-ecC!K z!Z#agp>zEW_H(P+DOMTh5@7*g%9yQF5w!ZUu!2pD*Yl~NV}!;r=5)~B^+ACwTH{~) z>AMJ)tes*XuVwN88IWx(&vnHWJ@^aG@kvhMI;f1ed{*=A%$AHx7w=(1E#8CHZ^rAa z?r!R@u36oafOn>IlR=l zjGJT@AE)-}^k#cuiUlWnU7cEo!AqN#ih`OMN<{&^c9kN3OnsmI{xU%7orr)9Mvpa& zzG$Vz3*fUG4qZ}w;Yr>6Kyh>_(_C0@vP~D^dQZN#t8hN&yT`0*f`$Zc1{L0+kkFd; z4uY4d#v}Q=7F?Q+^Qte~%xt)6o!Mjn1r^&(eJ<8@e%RupLO>9lJ}$-G5ts3M#?+hWzYefSHq*P97PzpU7jnj9M$< z3%dbp`hLphNxS0XH_QHPdOVOi@GV0aMgE z&+P8rs*!}fN{`Wst-bb-bHiVI2lT>=nnEvM)1-TGL&llL_6oML>MEUHl#o;}lfKj)-n^-E62dZG8(ag_91?VWOba_&$732I zC#lze?NWZL>EukiO-&v0cug8~!sMO#tk#V|vW@`r@KlSO($!vq$CA&d9YV`arP0jO zVih-zGt4JE37@3wVeN9t5l@j?XND*F*N)>=~*Nh1$9TRak^ZVI>&jQG^vBCt z{pYO6algXrBRuf6Aq-e88ieqmcO|sxjJ( zJLbV$&(=qDTF1;-S3OvUV_Te7t1$d(I|i6pa3PJCqDL|+a-^$BUo2levtD53eD4GF zQ~J2w1{hQD`gUOR>pmI%on6NIq|#&rHsf;aXZnK5Q$-hlNxD1SQF$76tIj9!!ebxK zw~0|EsWC51y|dfPJSS3~bf87U+JxH$=RS1witT+jgmJj*7!}|P!`Cmaqet}g zb_tsTdQzI)ZC))2fwCqkgnb+62%Qx^(dhVcn5Np3Bm_I#NSnlLw{Np!`BuwG*^@sw z}dM6e^QT(-2d*X9=|GWAYXY+as*c<})xctdLBBy1^r_o(@LGe~a z(c)Zvu4~CNNg+~8QUOyXFT(c*j=D|F_meK2^C zN~3@o6#aDBWU+BNBC%}InD`@|VwueM(bsOzD%Z=9i-P3P!-(K@3 zO9VyJldi_U|K0cRFmN6T2_6w;TOiz+n4QZ5q!@_+bbAwZGbez~E&?#?d48u3|Nag8 zuh8pTM&@RYM98i)z+M4DlvRKtp**^OrurVbkTecm)(F6PVR zPa7~85(B*DBN_@IzaQ1m#3BTT?cfk3sRpnrN$e1!08Hh`wJ0E;{iue<01@W1i%GC^WH1z~TF$&_F?pe>5itBLI};KttR~)Ggi3 zi0hTe9URUHC;~y;REP Date: Thu, 28 Oct 2021 09:12:14 -0700 Subject: [PATCH 004/277] Document ZFP_WITH_CUDA in config file --- zfp-config.cmake.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/zfp-config.cmake.in b/zfp-config.cmake.in index 204665264..87ceb5d2c 100644 --- a/zfp-config.cmake.in +++ b/zfp-config.cmake.in @@ -3,7 +3,8 @@ # It defines the following variables # ZFP_INCLUDE_DIRS - include directories for zfp # ZFP_LIBRARIES - libraries to link against -# ZFP_WITH_OPENMP - Indicates if the zfp library has been built with OpenMP support. +# ZFP_WITH_OPENMP - indicates if the zfp library has been built with OpenMP support +# ZFP_WITH_CUDA - indicates if the zfp library has been built with CUDA support # # And the following imported targets: # zfp::zfp From 99e526cd5a0ffb8e29e964c22754174de7aad837 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 28 Oct 2021 09:58:47 -0700 Subject: [PATCH 005/277] Document CUDA limitations --- docs/source/execution.rst | 36 ++++++++++++++++++++++++------------ docs/source/limitations.rst | 2 ++ 2 files changed, 26 insertions(+), 12 deletions(-) diff --git a/docs/source/execution.rst b/docs/source/execution.rst index 4ef8db5cb..81d066788 100644 --- a/docs/source/execution.rst +++ b/docs/source/execution.rst @@ -186,18 +186,30 @@ stream is copied back to the host and device memory is deallocated. If both pointers are device pointers, then no copies are made. Additionally, any combination of mixing host and device pointers is supported. -Additional Requirements -^^^^^^^^^^^^^^^^^^^^^^^ - -The CUDA implementation supports strided fields. However, when the field -is stored in host memory, it must occupy contiguous storage, i.e., with -no unused memory addresses between the minimum and maximum address spanned -by the field. This requirement avoids having to copy and allocate more -temporary memory than needed to hold the array if it were not strided. -Note that the strides can still be arbitrary as long as they serve only to -permute the array elements. Moreover, this restriction applies only to the -CUDA execution policy and the case where the uncompressed field resides on -the host. +.. _cuda-limitations: + +CUDA Limitations +^^^^^^^^^^^^^^^^ + +The CUDA implementation has a number of limitations: + +* Only the :ref:`fixed-rate mode ` mode is supported. + Other modes will be supported in a future release. +* 4D arrays are not supported. +* :ref:`Headers
` are not supported. Any header already present in + the stream will be silently overwritten on compression. +* |zfp| must be built with a :c:macro:`ZFP_BIT_STREAM_WORD_SIZE` of 64 bits. +* Although :ref:`strides ` are supported, fields must be contiguous + when stored in host memory, i.e., with no unused memory addresses between + the minimum and maximum address spanned by the field (see + :c:func:`zfp_field_is_contiguous`). This requirement avoids having to copy + and allocate more temporary memory than needed to hold the array if it were + not strided. Note that the strides can still be arbitrary as long as they + serve only to permute the array elements. Moreover, this restriction + applies only to the CUDA execution policy and the case where the + uncompressed field resides on the host. + +We expect to address these limitations over time. Setting the Execution Policy diff --git a/docs/source/limitations.rst b/docs/source/limitations.rst index 8f36f781c..eacd6cb5d 100644 --- a/docs/source/limitations.rst +++ b/docs/source/limitations.rst @@ -74,6 +74,8 @@ that will address some of these limitations. - Version |cudarelease| adds support for CUDA compression and decompression. However, only the fixed-rate compression mode is so far supported. + The CUDA implementation is further subject to + :ref:`additional limitations `. - As of version |4drelease|, |zfp| supports compression and decompression of 4D arrays. However, |zfp| does not yet implement a 4D compressed From 5c991daaeaf2da7c493148e4a618aa624f405839 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 3 Nov 2021 16:56:12 -0700 Subject: [PATCH 006/277] Add/rename top-level files for E4S compliance --- CHANGELOG.md | 388 ++++++++++++++++++++++++++++++++++++++++++++++++ CONTRIBUTING.md | 13 ++ README.md | 38 ++--- SUPPORT.md | 11 ++ VERSIONS.md | 298 ------------------------------------- 5 files changed, 425 insertions(+), 323 deletions(-) create mode 100644 CHANGELOG.md create mode 100644 CONTRIBUTING.md create mode 100644 SUPPORT.md delete mode 100644 VERSIONS.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..e37c1c015 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,388 @@ +Change Log +========== + +--- + +## Unreleased + +This future release is not ABI compatible with prior releases due to numerous +changes to function signatures and data structures like `zfp_field`. However, +few of the API changes, other than to cfp, should impact existing code. + +### Added + +- `zfp::const_array`: read-only variable-rate array that supports + fixed-precision, fixed-accuracy, and reversible modes. +- Compressed-array classes for 4D data. +- `const` versions of array references, pointers, and iterators. +- A more complete API for pointers and iterators. +- Support for pointers and iterators into array views. +- `zfp::array::size_bytes()` allows querying the size of different components + of an array object (e.g., payload, cache, index, metadata, ...). +- Templated C++ wrappers around the low-level C API. +- Additional functions for querying `zfp_field` and `zfp_stream` structs. +- `zfp_config`: struct that encapsulates compression mode and parameters. +- Rounding modes for reducing bias in compression errors. +- New examples: `ppm` and `iteratorC`. + +### Changed + +- `size_t` and `ptrdiff_t` replace `uint` and `int` for array sizes and + strides in the array classes and C API. +- `zfp_bool` replaces `int` as Boolean type in the C API. +- All array and view iterators are now random-access iterators. +- Array inspectors now return `const_reference` rather than a scalar + type like `float` to allow obtaining a `const_pointer` to an element + of an immutable array. +- `zfp::array::compressed_data()` now returns `void*` instead of `uchar*`. +- The array (de)serialization API has been revised, resulting in new + `zfp::array::header` and `zfp::exception` classes with new exception + messages. +- The array `codec` class is now responsible for all details regarding + compression. +- The compressed-array C++ implementation has been completely refactored to + make it more modular, extensible, and reusable across array types. +- Array block shapes are now computed on the fly rather than stored. +- The cfp API now wraps array objects in structs. +- The zfpy decompression API now supports the more general `memoryview` over + `bytes` objects. +- Some command-line options for the `diffusion` example have changed. +- CMake 3.9 or later is now required for CMake builds. + +### Removed + +- `zfp::array::get_header()` has been replaced with a `zfp::array::header` + constructor that accepts an array object. + +### Fixed + +- #66: `make install` overwrites googletest. +- #84: Incorrect order of parameters in CUDA `memset()`. +- #86: C++ compiler warns when `__STDC_VERSION__` is undefined. +- #87: `CXXFLAGS` is misspelled in `cfp/src/Makefile`. +- #98: `zfp_stream_maximum_size()` underestimates size in reversible mode. +- #99: Incorrect `private_view` reads due to missing writeback. +- #109: Unused CPython array is incompatible with PyPy. +- #112: PGI compiler bug causes issues with memory alignment. +- #119: All-subnormal blocks may cause floating-point overflow. +- #121: CUDA bit offsets are limited to 32 bits. +- #122: `make install` does not install zfp command-line utility. +- #125: OpenMP bit offsets are limited to 32 bits. +- #126: `make install` does not install Fortran module. +- #127: Reversible mode reports incorrect compressed block size. +- `ZFP_MAX_BITS` is off by one. +- `diffusionC`, `iteratorC` are not being built with `gmake`. + +--- + +## 0.5.5 (2019-05-05) + +### Added + +- Support for reversible (lossless) compression of floating-point and + integer data. +- Methods for serializing and deserializing zfp's compressed arrays. +- Python bindings for compressing NumPy arrays. +- Fortran bindings to zfp's high-level C API. + +### Changed + +- The default compressed-array cache size is now a function of the total + number of array elements, irrespective of array shape. + +### Fixed + +- Incorrect handling of execution policy in zfp utility. +- Incorrect handling of decompression via header in zfp utility. +- Incorrect cleanup of device memory in CUDA decompress. +- Missing tests for failing mallocs. +- CMake does not install CFP when built. +- `zfp_write_header()` and `zfp_field_metadata()` succeed even if array + dimensions are too large to fit in header. + +--- + +## 0.5.4 (2018-10-01) + +### Added + +- Support for CUDA fixed-rate compression and decompression. +- Views into compressed arrays for thread safety, nested array indexing, + slicing, and array subsetting. +- C language bindings for compressed arrays. +- Support for compressing and decompressing 4D data. + +### Changed + +- Execution policy now applies to both compression and decompression. +- Compressed array accessors now return Scalar type instead of + const Scalar& to avoid stale references to evicted cache lines. + +### Fixed + +- Incorrect handling of negative strides. +- Incorrect handling of arrays with more than 2^32 elements in zfp command-line + tool. +- `bitstream` is not C++ compatible. +- Minimum cache size request is not respected. + +--- + +## 0.5.3 (2018-03-28) + +### Added + +- Support for OpenMP multithreaded compression (but not decompression). +- Options for OpenMP execution in zfp command-line tool. +- Compressed-array support for copy construction and assignment via deep + copies. +- Virtual destructors to enable inheritance from zfp arrays. + +### Changed + +- `zfp_decompress()` now returns the number of compressed bytes processed so + far, i.e., the same value returned by `zfp_compress()`. + +--- + +## 0.5.2 (2017-09-28) + +### Added + +- Iterators and proxy objects for pointers and references. +- Example illustrating how to use iterators and pointers. + +### Changed + +- Diffusion example now optionally uses iterators. +- Moved internal headers under array to `array/zfp`. +- Modified 64-bit integer typedefs to avoid the C89 non-compliant `long long` + and allow for user-supplied types and literal suffixes. +- Renamed compile-time macros that did not have a `ZFP` prefix. +- Rewrote documentation in reStructuredText and added complete documentation + of all public functions, classes, types, and macros. + +### Fixed + +- Issue with setting stream word type via CMake. + +--- + +## 0.5.1 (2017-03-28) + +This release primarily fixes a few minor issues but also includes changes in +anticipation of a large number of planned future additions to the library. +No changes have been made to the compressed format, which is backwards +compatible with version 0.5.0. + +### Added + +- High-level API support for integer types. +- Example that illustrates in-place compression. +- Support for CMake builds. +- Documentation that discusses common issues with using zfp. + +### Changed + +- Separated library version from CODEC version and added version string. +- Corrected inconsistent naming of `BIT_STREAM` macros in code and + documentation. +- Renamed some of the header bit mask macros. +- `stream_skip()` and `stream_flush()` now return the number of bits skipped + or output. +- Renamed `stream_block()` and `stream_delta()` to make it clear that they + refer to strided streams. Added missing definition of + `stream_stride_block()`. +- Changed `int` and `uint` types in places to use `ptrdiff_t` and `size_t` + where appropriate. +- Changed API for `zfp_set_precision()` and `zfp_set_accuracy()` to not + require the scalar type. +- Added missing `static` keyword in `decode_block()`. +- Changed `testzfp` to allow specifying which tests to perform on the + command line. +- Modified directory structure. + +### Fixed + +- Bug that prevented defining uninitialized arrays. +- Incorrect computation of array sizes in `zfp_field_size()`. +- Minor issues that prevented code from compiling on Windows. +- Issue with fixed-accuracy headers that caused unnecessary storage. + +--- + +## 0.5.0 (2016-02-29) + +This version introduces backwards incompatible changes to the CODEC. + +### Added + +- Modified CODEC to more efficiently encode blocks whose values are all + zero or are smaller in magnitude than the absolute error tolerance. + This allows representing "empty" blocks using only one bit each. +- Added functions for compactly encoding the compression parameters + and field meta data, e.g., for producing self-contained compressed + streams. Also added functions for reading and writing a header + containing these parameters. + +### Changed + +- Changed behavior of `zfp_compress()` and `zfp_decompress()` to not + automatically rewind the bit stream. This makes it easier to concatenate + multiple compressed bit streams, e.g., when compressing vector fields or + multiple scalars together. +- Changed the zfp example program interface to allow reading and writing + compressed streams, optionally with a header. The zfp tool can now be + used to compress and decompress files as a stand alone utility. + +--- + +## 0.4.1 (2015-12-28) + +### Added + +- Added `simple.c` as a minimal example of how to call the compressor. + +### Changed + +- Changed compilation of diffusion example to output two executables: + one with and one without compression. + +### Fixed + +- Bug that caused segmentation fault when compressing 3D arrays whose + dimensions are not multiples of four. Specifically, arrays of dimensions + nx * ny * nz, with ny not a multiple of four, were not handled correctly. +- Modified `examples/fields.h` to ensure standard compliance. Previously, + C99 support was needed to handle the hex float constants, which are + not supported in C++98. + +--- + +## 0.4.0 (2015-12-05) + +This version contains substantial changes to the compression algorithm that +improve PSNR by about 6 dB and speed by a factor of 2-3. These changes are +not backward compatible with previous versions of zfp. + +### Added + +- Support for 31-bit and 63-bit integer data, as well as shorter integer types. +- New examples for evaluating the throughput of the (de)compressor and for + compressing grayscale images in the pgm format. +- Frequently asked questions. + +### Changed + +- Rewrote compression codec entirely in C to make linking and calling + easier from other programming languages, and to expose the low-level + interface through C instead of C++. This necessitated significant + changes to the API as well. +- Minor changes to the C++ compressed array API, as well as major + implementation changes to support the C library. The namespace and + public types are now all in lower case. + +### Removed + +- Support for general fixed-point decorrelating transforms. + +--- + +## 0.3.2 (2015-12-03) + +### Fixed + +- Bug in `Array::get()` that caused the wrong cached block to be looked up, + thus occasionally copying incorrect values back to parts of the array. + +--- + +## 0.3.1 (2015-05-06) + +### Fixed + +- Rare bug caused by exponent underflow in blocks with no normal and some + denormal numbers. + +--- + +## 0.3.0 (2015-03-03) + +This version modifies the default decorrelating transform to one that uses +only additions and bit shifts. This new transform, in addition to being +faster, also has some theoretical optimality properties and tends to improve +rate distortion. This change is not backwards compatible. + +### Added + +- Compile-time support for parameterized transforms, e.g., to support other + popular transforms like DCT, HCT, and Walsh-Hadamard. +- Floating-point traits to reduce the number of template parameters. It is + now possible to declare a 3D array as `Array3`, for example. +- Functions for setting the array scalar type and dimensions. +- `testzfp` for regression testing. + +### Changed + +- Made forward transform range preserving: (-1, 1) is mapped to (-1, 1). + Consequently Q1.62 fixed point can be used throughout. +- Changed the order in which bits are emitted within each bit plane to be more + intelligent. Group tests are now deferred until they are needed, i.e., just + before the value bits for the group being tested. This improves the quality + of fixed-rate encodings, but has no impact on compressed size. +- Made several optimizations to improve performance. +- Consolidated several header files. + +--- + +## 0.2.1 (2014-12-12) + +### Added + +- Win64 support via Microsoft Visual Studio compiler. +- Documentation of the expected output for the diffusion example. + +### Changed + +- Made several minor changes to suppress compiler warnings. + +### Fixed + +- Broken support for IBM's `xlc` compiler. + +--- + +## 0.2.0 (2014-12-02) + +The compression interface from `zfpcompress` was relocated to a separate +library, called `libzfp`, and modified to be callable from C. This API now +uses a parameter object (`zfp_params`) to specify array type and dimensions +as well as compression parameters. + +### Added + +- Several utility functions were added to simplify `libzfp` usage: + * Functions for setting the rate, precision, and accuracy. + Corresponding functions were also added to the `Codec` class. + * A function for estimating the buffer size needed for compression. +- The `Array` class functionality was expanded: + * Support for accessing the compressed bit stream stored with an array, + e.g., for offline compressed storage and for initializing an already + compressed array. + * Functions for dynamically specifying the cache size. + * The default cache is now direct-mapped instead of two-way associative. + +### Fixed + +- Corrected the value of the lowest possible bit plane to account for both + the smallest exponent and the number of bits in the significand. +- Corrected inconsistent use of rate and precision. The rate refers to the + number of compressed bits per floating-point value, while the precision + refers to the number of uncompressed bits. The `Array` API was changed + accordingly. + +--- + +## 0.1.0 (2014-11-12) + +Initial beta release. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..9bc8fa832 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,13 @@ +Contributing +============ + +The zfp project uses the +[Gitflow](https://nvie.com/posts/a-successful-git-branching-model/) +development model. Contributions should be made as pull requests on the +`develop` branch. Although this branch is under continuous development, +it should be robust enough to pass all regression tests. For contributions +that are not production ready, please [contact us](mailto:zfp.llnl.gov) to +have a separate branch created. The `master` branch is updated with each +release and reflects the most recent official release of zfp. See the +[Releases Page](https://github.com/LLNL/zfp/releases) for a history +of releases. diff --git a/README.md b/README.md index c9c3a1e92..34ae2de04 100644 --- a/README.md +++ b/README.md @@ -66,19 +66,11 @@ Full HTML [documentation](http://zfp.readthedocs.io/) is available online. A [PDF](http://readthedocs.org/projects/zfp/downloads/pdf/latest/) version is also available. +Further information on the zfp software is included in these files: -Contributing ------------- - -The zfp project uses the -[Gitflow](https://nvie.com/posts/a-successful-git-branching-model/) -development model. Contributions should be made as pull requests on the -`develop` branch. Although this branch is under continuous development, -it should be robust enough to pass all regression tests. -The `master` branch is updated with each release and reflects the most -recent official release of zfp. See the -[Releases Page](https://github.com/LLNL/zfp/releases) for a history -of releases. +- Change log: see [CHANGELOG.md](./CHANGELOG.md). +- Support and additional resources: see [SUPPORT.md](./SUPPORT.md). +- Code contributions: see [CONTRIBUTING.md](./CONTRIBUTING.md). Authors @@ -98,25 +90,21 @@ If you use zfp for scholarly research, please cite this paper: IEEE Transactions on Visualization and Computer Graphics, 20(12):2674-2683, December 2014. [doi:10.1109/TVCG.2014.2346458](http://doi.org/10.1109/TVCG.2014.2346458). +The algorithm implemented in the current version of zfp is described in the +[documentation](https://zfp.readthedocs.io/en/latest/algorithm.html) and in +the following paper: -Additional Resources --------------------- - -For more information on zfp, please see the -[zfp website](https://computing.llnl.gov/casc/zfp/). -For bug reports and feature requests, please consult the -[GitHub issue tracker](https://github.com/LLNL/zfp/issues/). -For questions and comments not answered here or in the -[documentation](http://zfp.readthedocs.io), -please send e-mail to [zfp@llnl.gov](mailto:zfp@llnl.gov). +* James Diffenderfer, Alyson Fox, Jeffrey Hittinger, Geoffrey Sanders, Peter Lindstrom. + [Error Analysis of ZFP Compression for Floating-Point Data](https://www.researchgate.net/publication/324908266_Error_Analysis_of_ZFP_Compression_for_Floating-Point_Data). + SIAM Journal on Scientific Computing, 41(3):A1867-A1898, June 2019. + [doi:10.1137/18M1168832](http://doi.org/10.1137/18M1168832). License ------- -zfp is distributed under the terms of the BSD 3-Clause license. See the -files [LICENSE](https://github.com/LLNL/zfp/blob/develop/LICENSE) and -[NOTICE](https://github.com/LLNL/zfp/blob/develop/NOTICE) for details. +zfp is distributed under the terms of the BSD 3-Clause license. See +[LICENSE](./LICENSE) and [NOTICE](./NOTICE) for details. SPDX-License-Identifier: BSD-3-Clause diff --git a/SUPPORT.md b/SUPPORT.md new file mode 100644 index 000000000..83a979317 --- /dev/null +++ b/SUPPORT.md @@ -0,0 +1,11 @@ +Support +======= + +For more information on zfp, please see the +[zfp website](https://zfp.llnl.gov). +For bug reports and feature requests, please consult the +[GitHub issue tracker](https://github.com/LLNL/zfp/issues/). +For questions and comments not answered here or in the +[documentation](http://zfp.readthedocs.io), +please contact us by email at +[zfp@llnl.gov](mailto:zfp@llnl.gov). diff --git a/VERSIONS.md b/VERSIONS.md deleted file mode 100644 index 2b7e0e72e..000000000 --- a/VERSIONS.md +++ /dev/null @@ -1,298 +0,0 @@ -# zfp Release Notes - -## 0.5.5 (May 5, 2019) - -- Added support for reversible (lossless) compression of floating-point and - integer data. - -- Added methods for serializing and deserializing zfp's compressed arrays. - -- Added Python bindings for compressing NumPy arrays. - -- Added Fortran bindings to zfp's high-level C API. - -- Change: - - The default compressed-array cache size is now a function of the total - number of array elements, irrespective of array shape. - -- Bug fixes: - - Incorrect handling of execution policy in zfp utility. - - Incorrect handling of decompression via header in zfp utility. - - Incorrect cleanup of device memory in CUDA decompress. - - Tests for failing mallocs. - - CMake installation of CFP when built. - - zfp\_write\_header and zfp\_field\_metadata now fail if array dimensions - are too large to fit in header. - - -## 0.5.4 (October 1, 2018) - -- Added support for CUDA fixed-rate compression and decompression. - -- Added views into compressed arrays for thread safety, nested array - indexing, slicing, and array subsetting. - -- Added C language bindings for compressed arrays. - -- Added support for compressing and decompressing 4D data. - -- Changes: - - Execution policy now applies to both compression and decompression. - - Compressed array accessors now return Scalar type instead of - const Scalar& to avoid stale references to evicted cache lines. - -- Bug fixes: - - Handling of negative strides. - - Command line tool handling of arrays with more than 2^32 elements. - - bitstream C++ compatibility. - - Respect minimum cache size request. - - -## 0.5.3 (March 28, 2018) - -- Added support for OpenMP multithreaded compression (but not decompression). - -- Added options for OpenMP execution to zfp command-line tool. - -- Changed return value of zfp\_decompress to indicate the number of compressed - bytes processed so far (now returns same value as zfp\_compress on success). - -- Added compressed array support for copy construction and assignment via - deep copies. - -- Added virtual destructors to enable inheritance from zfp arrays. - - -## 0.5.2 (September 28, 2017) - -- Added iterators and proxy objects for pointers and references. - -- Added example illustrating how to use iterators and pointers. - -- Modified diffusion example to optionally use iterators. - -- Moved internal headers under array to array/zfp. - -- Modified 64-bit integer typedefs to avoid the C89 non-compliant long long - and allow for user-supplied types and literal suffixes. - -- Renamed compile-time macros that did not have a ZFP prefix. - -- Fixed issue with setting stream word type via CMake. - -- Rewrote documentation in reStructuredText and added complete - documentation of all public functions, classes, types, and macros. - Removed ASCII documentation. - - -## 0.5.1 (March 28, 2017) - -- This release primarily fixes a few minor issues but also includes - changes in anticipation of a large number of planned future additions - to the library. No changes have been made to the compressed format, - which is backwards compatible with version 0.5.0. - -- Added high-level API support for integer types. - -- Separated library version from CODEC version and added version string. - -- Added example that illustrates in-place compression. - -- Added support for CMake builds. - -- Corrected inconsistent naming of BIT\_STREAM macros in code and - documentation. - -- Renamed some of the header bit mask macros. - -- Added return values to stream\_skip and stream\_flush to indicate the - number of bits skipped or output. - -- Renamed stream\_block and stream\_delta to make it clear that they refer - to strided streams. Added missing definition of stream\_stride\_block. - -- Changed int/uint types in places to use ptrdiff\_t/size\_t where - appropriate. - -- Changed API for zfp\_set\_precision and zfp\_set\_accuracy to not require - the scalar type. - -- Added missing static keyword in decode\_block. - -- Changed testzfp to allow specifying which tests to perform on the - command line. - -- Fixed bug that prevented defining uninitialized arrays. - -- Fixed incorrect computation of array sizes in zfp\_field\_size. - -- Fixed minor issues that prevented code from compiling on Windows. - -- Fixed issue with fixed-accuracy headers that caused unnecessary storage. - -- Modified directory structure. - -- Added documentation that discusses common issues with using zfp. - - -## 0.5.0 (February 29, 2016) - -- Modified CODEC to more efficiently encode blocks whose values are all - zero or are smaller in magnitude than the absolute error tolerance. - This allows representing "empty" blocks using only one bit each. This - version is not backwards compatible with prior zfp versions. - -- Changed behavior of zfp\_compress and zfp\_decompress to not automatically - rewind the bit stream. This makes it easier to concatenate multiple - compressed bit streams, e.g., when compressing vector fields or multiple - scalars together. - -- Added functions for compactly encoding the compression parameters - and field meta data, e.g., for producing self-contained compressed - streams. Also added functions for reading and writing a header - containing these parameters. - -- Changed the zfp example program interface to allow reading and writing - compressed streams, optionally with a header. The zfp tool can now be - used to compress and decompress files as a stand alone utility. - - -## 0.4.1 (December 28, 2015) - -- Fixed bug that caused segmentation fault when compressing 3D arrays - whose dimensions are not multiples of four. Specifically, arrays of - dimensions nx * ny * nz, with ny not a multiple of four, were not - handled correctly. - -- Modified examples/fields.h to ensure standard compliance. Previously, - C99 support was needed to handle the hex float constants, which are - not supported in C++98. - -- Added simple.c as a minimal example of how to call the compressor. - -- Changed compilation of diffusion example to output two executables: - one with and one without compression. - - -## 0.4.0 (December 5, 2015) - -- Substantial changes to the compression algorithm that improve PSNR - by about 6 dB and speed by a factor of 2-3. These changes are not - backward compatible with previous versions of zfp. - -- Added support for 31-bit and 63-bit integer data, as well as shorter - integer types. - -- Rewrote compression codec entirely in C to make linking and calling - easier from other programming languages, and to expose the low-level - interface through C instead of C++. This necessitated significant - changes to the API as well. - -- Minor changes to the C++ compressed array API, as well as major - implementation changes to support the C library. The namespace and - public types are now all in lower case. - -- Deprecated support for general fixed-point decorrelating transforms - and slimmed down implementation. - -- Added new examples for evaluating the throughput of the (de)compressor - and for compressing grayscale images in the pgm format. - -- Added FAQ. - - -## 0.3.2 (December 3, 2015) - -- Fixed bug in Array::get() that caused the wrong cached block to be - looked up, thus occasionally copying incorrect values back to parts - of the array. - - -## 0.3.1 (May 6, 2015) - -- Fixed rare bug caused by exponent underflow in blocks with no normal - and some denormal numbers. - - -## 0.3.0 (March 3, 2015) - -- Modified the default decorrelating transform to one that uses only - additions and bit shifts. This new transform, in addition to being - faster, also has some theoretical optimality properties and tends to - improve rate distortion. - -- Added compile-time support for parameterized transforms, e.g., to - support other popular transforms like DCT, HCT, and Walsh-Hadamard. - -- Made forward transform range preserving: (-1, 1) is mapped to (-1, 1). - Consequently Q1.62 fixed point can be used throughout. - -- Changed the order in which bits are emitted within each bit plane - to be more intelligent. Group tests are now deferred until they - are needed, i.e., just before the value bits for the group being - tested. This improves the quality of fixed-rate encodings, but - has no impact on compressed size. - -- Made several optimizations to improve performance. - -- Added floating-point traits to reduce the number of template - parameters. It is now possible to declare a 3D array as - Array3, for example. - -- Added functions for setting the array scalar type and dimensions. - -- Consolidated several header files. - -- Added testzfp for regression testing. - - -## 0.2.1 (December 12, 2014) - -- Added Win64 support via Microsoft Visual Studio compiler. - -- Fixed broken support for IBM's xlc compiler. - -- Made several minor changes to suppress compiler warnings. - -- Documented expected output for the diffusion example. - - -## 0.2.0 (December 2, 2014) - -- The compression interface from zfpcompress was relocated to a - separate library, called libzfp, and modified to be callable from C. - This API now uses a parameter object (zfp\_params) to specify array - type and dimensions as well as compression parameters. - -- Several utility functions were added to simplify libzfp usage: - - * Functions for setting the rate, precision, and accuracy. - Corresponding functions were also added to the Codec class. - - * A function for estimating the buffer size needed for compression. - -- The Array class functionality was expanded: - - * Support for accessing the compressed bit stream stored with an - array, e.g., for offline compressed storage and for initializing - an already compressed array. - - * Functions for dynamically specifying the cache size. - - * The default cache is now direct-mapped instead of two-way - associative. - -- Minor bug fixes: - - * Corrected the value of the lowest possible bit plane to account for - both the smallest exponent and the number of bits in the significand. - - * Corrected inconsistent use of rate and precision. The rate refers - to the number of compressed bits per floating-point value, while - the precision refers to the number of uncompressed bits. The Array - API was changed accordingly. - - -## 0.1.0 (November 12, 2014) - -- Initial beta release. From d65af032c95c3403a912c80988b10774fb4f4b17 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 3 Nov 2021 16:56:56 -0700 Subject: [PATCH 007/277] Add link to MVAPICH2-GDR --- docs/source/introduction.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index 8e9852747..90a90ba0e 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -82,6 +82,9 @@ plugins, and formats, such as * `Compression functions `__ for Intel\ |reg| `Integrated Performance Primitives `__. +* `Compressed MPI messages `__ + in `MVAPICH2-GDR `__. + * `Compression CODEC `__ underlying the `OpenZGY `__ From de381394fc2e82753ca6a6c828a91f42d88ea2b7 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Mon, 8 Nov 2021 11:04:56 -0800 Subject: [PATCH 008/277] Fix unused variable warning --- .../array/decode/testTemplatedDecodeBase.cpp | 42 +++++++++---------- 1 file changed, 20 insertions(+), 22 deletions(-) diff --git a/tests/array/decode/testTemplatedDecodeBase.cpp b/tests/array/decode/testTemplatedDecodeBase.cpp index 21c995d86..9a2d7d41e 100644 --- a/tests/array/decode/testTemplatedDecodeBase.cpp +++ b/tests/array/decode/testTemplatedDecodeBase.cpp @@ -142,10 +142,9 @@ void populateStridedArray(SCALAR** dataArr, SCALAR dummyVal) void assertStridedBlockEntriesEqual(SCALAR* data1, SCALAR* data2) { - size_t i, j, k, l, countX, countY, countZ, countW; switch (DIMS) { case 1: - countX = BLOCK_SIDE_LEN * SX; + size_t countX = BLOCK_SIDE_LEN * SX; for (size_t i = 0; i < countX; i++) { if (!(i % (countX/BLOCK_SIDE_LEN))) { @@ -157,8 +156,8 @@ void assertStridedBlockEntriesEqual(SCALAR* data1, SCALAR* data2) break; case 2: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; for (size_t j = 0; j < countY; j++) { for (size_t i = 0; i < countX; i++) { @@ -173,9 +172,9 @@ void assertStridedBlockEntriesEqual(SCALAR* data1, SCALAR* data2) break; case 3: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - countZ = SZ / SY; + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; for (size_t k = 0; k < countZ; k++) { for (size_t j = 0; j < countY; j++) { @@ -195,10 +194,10 @@ void assertStridedBlockEntriesEqual(SCALAR* data1, SCALAR* data2) break; case 4: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - countZ = SZ / SY; - countW = SW / SZ; + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + size_t countW = SW / SZ; for (size_t l = 0; l < countW; l++) { for (size_t k = 0; k < countZ; k++) { @@ -224,10 +223,9 @@ void assertStridedBlockEntriesEqual(SCALAR* data1, SCALAR* data2) void assertPartialBlockEntriesEqual(SCALAR* data1, SCALAR* data2) { - size_t i, j, k, l, countX, countY, countZ, countW; switch (DIMS) { case 1: - countX = BLOCK_SIDE_LEN * SX; + size_t countX = BLOCK_SIDE_LEN * SX; for (size_t i = 0; i < countX; i++) { if (i/(countX/BLOCK_SIDE_LEN) < PX @@ -240,8 +238,8 @@ void assertPartialBlockEntriesEqual(SCALAR* data1, SCALAR* data2) break; case 2: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; for (size_t j = 0; j < countY; j++) { for (size_t i = 0; i < countX; i++) { @@ -258,9 +256,9 @@ void assertPartialBlockEntriesEqual(SCALAR* data1, SCALAR* data2) break; case 3: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - countZ = SZ / SY; + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; for (size_t k = 0; k < countZ; k++) { for (size_t j = 0; j < countY; j++) { @@ -283,10 +281,10 @@ void assertPartialBlockEntriesEqual(SCALAR* data1, SCALAR* data2) break; case 4: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - countZ = SZ / SY; - countW = SW / SZ; + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + size_t countW = SW / SZ; for (size_t l = 0; l < countW; l++) { for (size_t k = 0; k < countZ; k++) { From 6aec9dc420bc692873c9b3e270d7c23e4f61d2fa Mon Sep 17 00:00:00 2001 From: gmorriso Date: Mon, 8 Nov 2021 12:31:47 -0800 Subject: [PATCH 009/277] compilation error fix --- .../array/decode/testTemplatedDecodeBase.cpp | 432 ++++++++---------- 1 file changed, 203 insertions(+), 229 deletions(-) diff --git a/tests/array/decode/testTemplatedDecodeBase.cpp b/tests/array/decode/testTemplatedDecodeBase.cpp index 9a2d7d41e..35873a52b 100644 --- a/tests/array/decode/testTemplatedDecodeBase.cpp +++ b/tests/array/decode/testTemplatedDecodeBase.cpp @@ -35,281 +35,255 @@ void populateArray(SCALAR** dataArr) void populateStridedArray(SCALAR** dataArr, SCALAR dummyVal) { - size_t i, j, k, l, countX, countY, countZ, countW; - - - switch(DIMS) { - case 1: - countX = BLOCK_SIDE_LEN * SX; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX); - ASSERT_TRUE(*dataArr != nullptr); +#if DIMS == 1 + size_t countX = BLOCK_SIDE_LEN * SX; + *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX); + ASSERT_TRUE(*dataArr != nullptr); - for (i = 0; i < countX; i++) { - if (i % SX) { - (*dataArr)[i] = dummyVal; - } else { + for (size_t i = 0; i < countX; i++) { + if (i % SX) { + (*dataArr)[i] = dummyVal; + } else { #ifdef FL_PT_DATA - (*dataArr)[i] = nextSignedRandFlPt(); + (*dataArr)[i] = nextSignedRandFlPt(); #else - (*dataArr)[i] = nextSignedRandInt(); + (*dataArr)[i] = nextSignedRandInt(); #endif - } - } - break; + } + } - case 2: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY); - ASSERT_TRUE(*dataArr != nullptr); +#elif DIMS == 2 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY); + ASSERT_TRUE(*dataArr != nullptr); - for (j = 0; j < countY; j++) { - for (i = 0; i < countX; i++) { - size_t index = countX*j + i; - if (i % (countX/BLOCK_SIDE_LEN) - || j % (countY/BLOCK_SIDE_LEN)) { - (*dataArr)[index] = dummyVal; - } else { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + size_t index = countX*j + i; + if (i % (countX/BLOCK_SIDE_LEN) + || j % (countY/BLOCK_SIDE_LEN)) { + (*dataArr)[index] = dummyVal; + } else { #ifdef FL_PT_DATA - (*dataArr)[index] = nextSignedRandFlPt(); + (*dataArr)[index] = nextSignedRandFlPt(); #else - (*dataArr)[index] = nextSignedRandInt(); + (*dataArr)[index] = nextSignedRandInt(); #endif - } - } } - break; - - case 3: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - countZ = SZ / SY; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ); - ASSERT_TRUE(*dataArr != nullptr); - - for (k = 0; k < countZ; k++) { - for (j = 0; j < countY; j++) { - for (i = 0; i < countX; i++) { - size_t index = countX*countY*k + countX*j + i; - if (i % (countX/BLOCK_SIDE_LEN) - || j % (countY/BLOCK_SIDE_LEN) - || k % (countZ/BLOCK_SIDE_LEN)) { - (*dataArr)[index] = dummyVal; - } else { + } + } + +#elif DIMS == 3 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ); + ASSERT_TRUE(*dataArr != nullptr); + + for (size_t k = 0; k < countZ; k++) { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + size_t index = countX*countY*k + countX*j + i; + if (i % (countX/BLOCK_SIDE_LEN) + || j % (countY/BLOCK_SIDE_LEN) + || k % (countZ/BLOCK_SIDE_LEN)) { + (*dataArr)[index] = dummyVal; + } else { #ifdef FL_PT_DATA - (*dataArr)[index] = nextSignedRandFlPt(); + (*dataArr)[index] = nextSignedRandFlPt(); #else - (*dataArr)[index] = nextSignedRandInt(); + (*dataArr)[index] = nextSignedRandInt(); #endif - } - } } } - break; - - case 4: - countX = BLOCK_SIDE_LEN * SX; - countY = SY / SX; - countZ = SZ / SY; - countW = SW / SZ; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ * countW); - ASSERT_TRUE(*dataArr != nullptr); - - for (l = 0; l < countW; l++) { - for (k = 0; k < countZ; k++) { - for (j = 0; j < countY; j++) { - for (i = 0; i < countX; i++) { - size_t index = countX*countY*countZ*l + countX*countY*k + countX*j + i; - if (i % (countX/BLOCK_SIDE_LEN) - || j % (countY/BLOCK_SIDE_LEN) - || k % (countZ/BLOCK_SIDE_LEN) - || l % (countW/BLOCK_SIDE_LEN)) { - (*dataArr)[index] = dummyVal; - } else { + } + } + +#elif DIMS == 4 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + size_t countW = SW / SZ; + *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ * countW); + ASSERT_TRUE(*dataArr != nullptr); + + for (size_t l = 0; l < countW; l++) { + for (size_t k = 0; k < countZ; k++) { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + size_t index = countX*countY*countZ*l + countX*countY*k + countX*j + i; + if (i % (countX/BLOCK_SIDE_LEN) + || j % (countY/BLOCK_SIDE_LEN) + || k % (countZ/BLOCK_SIDE_LEN) + || l % (countW/BLOCK_SIDE_LEN)) { + (*dataArr)[index] = dummyVal; + } else { #ifdef FL_PT_DATA - (*dataArr)[index] = nextSignedRandFlPt(); + (*dataArr)[index] = nextSignedRandFlPt(); #else - (*dataArr)[index] = nextSignedRandInt(); + (*dataArr)[index] = nextSignedRandInt(); #endif - } - } } } } - break; + } } +#endif } void assertStridedBlockEntriesEqual(SCALAR* data1, SCALAR* data2) { - switch (DIMS) { - case 1: - size_t countX = BLOCK_SIDE_LEN * SX; - - for (size_t i = 0; i < countX; i++) { - if (!(i % (countX/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[i], data2[i]) << - "index " << i << " mismatch: " << data1[i] << " != " << data2[i]; - } - } - - break; - - case 2: - size_t countX = BLOCK_SIDE_LEN * SX; - size_t countY = SY / SX; - - for (size_t j = 0; j < countY; j++) { - for (size_t i = 0; i < countX; i++) { - if (!(i % (countX/BLOCK_SIDE_LEN)) - && !(j % (countY/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[countX*j + i], data2[countX*j + i]) << - "index " << (countX*j + i) << " mismatch: " << data1[countX*j + i] << " != " << data2[countX*j + i]; - } - } - } +#if DIMS == 1 + size_t countX = BLOCK_SIDE_LEN * SX; - break; + for (size_t i = 0; i < countX; i++) { + if (!(i % (countX/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[i], data2[i]) << + "index " << i << " mismatch: " << data1[i] << " != " << data2[i]; + } + } - case 3: - size_t countX = BLOCK_SIDE_LEN * SX; - size_t countY = SY / SX; - size_t countZ = SZ / SY; +#elif DIMS == 2 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + if (!(i % (countX/BLOCK_SIDE_LEN)) + && !(j % (countY/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[countX*j + i], data2[countX*j + i]) << + "index " << (countX*j + i) << " mismatch: " << data1[countX*j + i] << " != " << data2[countX*j + i]; + } + } + } - for (size_t k = 0; k < countZ; k++) { - for (size_t j = 0; j < countY; j++) { - for (size_t i = 0; i < countX; i++) { - if (!(i % (countX/BLOCK_SIDE_LEN)) - && !(j % (countY/BLOCK_SIDE_LEN)) - && !(k % (countZ/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[countX*countY*k + countX*j + i], data2[countX*countY*k + countX*j + i]) << - "index " << (countX*countY*k + countX*j + i) << " mismatch: " << - data1[countX*countY*k + countX*j + i] << " != " << - data2[countX*countY*k + countX*j + i]; - } - } - } +#elif DIMS == 3 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + + for (size_t k = 0; k < countZ; k++) { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + if (!(i % (countX/BLOCK_SIDE_LEN)) + && !(j % (countY/BLOCK_SIDE_LEN)) + && !(k % (countZ/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[countX*countY*k + countX*j + i], data2[countX*countY*k + countX*j + i]) << + "index " << (countX*countY*k + countX*j + i) << " mismatch: " << + data1[countX*countY*k + countX*j + i] << " != " << + data2[countX*countY*k + countX*j + i]; } + } + } + } - break; - - case 4: - size_t countX = BLOCK_SIDE_LEN * SX; - size_t countY = SY / SX; - size_t countZ = SZ / SY; - size_t countW = SW / SZ; - - for (size_t l = 0; l < countW; l++) { - for (size_t k = 0; k < countZ; k++) { - for (size_t j = 0; j < countY; j++) { - for (size_t i = 0; i < countX; i++) { - if (!(i % (countX/BLOCK_SIDE_LEN)) - && !(j % (countY/BLOCK_SIDE_LEN)) - && !(k % (countZ/BLOCK_SIDE_LEN)) - && !(l % (countW/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[countX*countY*countZ*l + countX*countY*k + countX*j + i], data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]) << - "index " << (countX*countY*countZ*l + countX*countY*k + countX*j + i) << " mismatch: " << - data1[countX*countY*countZ*l + countX*countY*k + countX*j + i] << " != " << - data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]; - } - } - } +#elif DIMS == 4 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + size_t countW = SW / SZ; + + for (size_t l = 0; l < countW; l++) { + for (size_t k = 0; k < countZ; k++) { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + if (!(i % (countX/BLOCK_SIDE_LEN)) + && !(j % (countY/BLOCK_SIDE_LEN)) + && !(k % (countZ/BLOCK_SIDE_LEN)) + && !(l % (countW/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[countX*countY*countZ*l + countX*countY*k + countX*j + i], data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]) << + "index " << (countX*countY*countZ*l + countX*countY*k + countX*j + i) << " mismatch: " << + data1[countX*countY*countZ*l + countX*countY*k + countX*j + i] << " != " << + data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]; } } - - break; + } + } } +#endif } void assertPartialBlockEntriesEqual(SCALAR* data1, SCALAR* data2) { - switch (DIMS) { - case 1: - size_t countX = BLOCK_SIDE_LEN * SX; - - for (size_t i = 0; i < countX; i++) { - if (i/(countX/BLOCK_SIDE_LEN) < PX - && !(i % (countX/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[i], data2[i]) << - "index " << i << " mismatch: " << data1[i] << " != " << data2[i]; - } - } - - break; - - case 2: - size_t countX = BLOCK_SIDE_LEN * SX; - size_t countY = SY / SX; - - for (size_t j = 0; j < countY; j++) { - for (size_t i = 0; i < countX; i++) { - if (i/(countX/BLOCK_SIDE_LEN) < PX - && j/(countY/BLOCK_SIDE_LEN) < PY - && !(i % (countX/BLOCK_SIDE_LEN)) - && !(j % (countY/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[countX*j + i], data2[countX*j + i]) << - "index " << (countX*j + i) << " mismatch: " << data1[countX*j + i] << " != " << data2[countX*j + i]; - } - } - } +#if DIMS == 1 + size_t countX = BLOCK_SIDE_LEN * SX; - break; + for (size_t i = 0; i < countX; i++) { + if (i/(countX/BLOCK_SIDE_LEN) < PX + && !(i % (countX/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[i], data2[i]) << + "index " << i << " mismatch: " << data1[i] << " != " << data2[i]; + } + } - case 3: - size_t countX = BLOCK_SIDE_LEN * SX; - size_t countY = SY / SX; - size_t countZ = SZ / SY; +#elif DIMS == 2 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + if (i/(countX/BLOCK_SIDE_LEN) < PX + && j/(countY/BLOCK_SIDE_LEN) < PY + && !(i % (countX/BLOCK_SIDE_LEN)) + && !(j % (countY/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[countX*j + i], data2[countX*j + i]) << + "index " << (countX*j + i) << " mismatch: " << data1[countX*j + i] << " != " << data2[countX*j + i]; + } + } + } - for (size_t k = 0; k < countZ; k++) { - for (size_t j = 0; j < countY; j++) { - for (size_t i = 0; i < countX; i++) { - if (i/(countX/BLOCK_SIDE_LEN) < PX - && j/(countY/BLOCK_SIDE_LEN) < PY - && k/(countZ/BLOCK_SIDE_LEN) < PZ - && !(i % (countX/BLOCK_SIDE_LEN)) - && !(j % (countY/BLOCK_SIDE_LEN)) - && !(k % (countZ/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[countX*countY*k + countX*j + i], data2[countX*countY*k + countX*j + i]) << - "index " << (countX*countY*k + countX*j + i) << " mismatch: " << - data1[countX*countY*k + countX*j + i] << " != " << - data2[countX*countY*k + countX*j + i]; - } - } - } +#elif DIMS == 3 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + + for (size_t k = 0; k < countZ; k++) { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + if (i/(countX/BLOCK_SIDE_LEN) < PX + && j/(countY/BLOCK_SIDE_LEN) < PY + && k/(countZ/BLOCK_SIDE_LEN) < PZ + && !(i % (countX/BLOCK_SIDE_LEN)) + && !(j % (countY/BLOCK_SIDE_LEN)) + && !(k % (countZ/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[countX*countY*k + countX*j + i], data2[countX*countY*k + countX*j + i]) << + "index " << (countX*countY*k + countX*j + i) << " mismatch: " << + data1[countX*countY*k + countX*j + i] << " != " << + data2[countX*countY*k + countX*j + i]; } + } + } + } - break; - - case 4: - size_t countX = BLOCK_SIDE_LEN * SX; - size_t countY = SY / SX; - size_t countZ = SZ / SY; - size_t countW = SW / SZ; - - for (size_t l = 0; l < countW; l++) { - for (size_t k = 0; k < countZ; k++) { - for (size_t j = 0; j < countY; j++) { - for (size_t i = 0; i < countX; i++) { - if (i/(countX/BLOCK_SIDE_LEN) < PX - && j/(countY/BLOCK_SIDE_LEN) < PY - && k/(countZ/BLOCK_SIDE_LEN) < PZ - && l/(countW/BLOCK_SIDE_LEN) < PW - && !(i % (countX/BLOCK_SIDE_LEN)) - && !(j % (countY/BLOCK_SIDE_LEN)) - && !(k % (countZ/BLOCK_SIDE_LEN)) - && !(l % (countW/BLOCK_SIDE_LEN))) { - ASSERT_SCALAR_EQ(data1[countX*countY*countZ*l + countX*countY*k + countX*j + i], data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]) << - "index " << (countX*countY*countZ*l + countX*countY*k + countX*j + i) << " mismatch: " << - data1[countX*countY*countZ*l + countX*countY*k + countX*j + i] << " != " << - data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]; - } - } - } +#elif DIMS == 4 + size_t countX = BLOCK_SIDE_LEN * SX; + size_t countY = SY / SX; + size_t countZ = SZ / SY; + size_t countW = SW / SZ; + + for (size_t l = 0; l < countW; l++) { + for (size_t k = 0; k < countZ; k++) { + for (size_t j = 0; j < countY; j++) { + for (size_t i = 0; i < countX; i++) { + if (i/(countX/BLOCK_SIDE_LEN) < PX + && j/(countY/BLOCK_SIDE_LEN) < PY + && k/(countZ/BLOCK_SIDE_LEN) < PZ + && l/(countW/BLOCK_SIDE_LEN) < PW + && !(i % (countX/BLOCK_SIDE_LEN)) + && !(j % (countY/BLOCK_SIDE_LEN)) + && !(k % (countZ/BLOCK_SIDE_LEN)) + && !(l % (countW/BLOCK_SIDE_LEN))) { + ASSERT_SCALAR_EQ(data1[countX*countY*countZ*l + countX*countY*k + countX*j + i], data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]) << + "index " << (countX*countY*countZ*l + countX*countY*k + countX*j + i) << " mismatch: " << + data1[countX*countY*countZ*l + countX*countY*k + countX*j + i] << " != " << + data2[countX*countY*countZ*l + countX*countY*k + countX*j + i]; } } - - break; + } + } } +#endif } void setupStream(zfp_field** field, zfp_stream** stream, bool isStrided = false) From b91dcafb4bf844565f82676f90c189ef68f9aa5c Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 9 Nov 2021 09:49:26 -0800 Subject: [PATCH 010/277] Added field_metadata failure tests --- tests/src/misc/testZfpHeader.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/tests/src/misc/testZfpHeader.c b/tests/src/misc/testZfpHeader.c index b7e450f72..3ced7c45e 100644 --- a/tests/src/misc/testZfpHeader.c +++ b/tests/src/misc/testZfpHeader.c @@ -148,6 +148,35 @@ when_zfpFieldSetMetadataCalled_expect_arrayDimensionsSet(void **state) assert_int_equal(field->nz, 0); } +static void +when_zfpFieldMetadataCalled_onInvalidSize_expect_ZFP_META_NULL(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + uint64 metadata = zfp_field_metadata(field); + + field->nx = 1 << 25; + field->ny = 1 << 25; + + uint64 meta = zfp_field_metadata(field); + + assert_int_equal(meta, ZFP_META_NULL); +} + +static void +when_zfpFieldSetMetadataCalled_forInvalidMeta_expect_false(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + uint64 metadata = zfp_field_metadata(field); + + uint64 meta = 1ULL << (ZFP_META_BITS + 1); + + zfp_bool status = zfp_field_set_metadata(field, meta); + + assert_int_equal(status, zfp_false); +} + static void when_zfpWriteHeaderMagic_expect_numBitsWrittenEqualToZFP_MAGIC_BITS(void **state) { @@ -436,6 +465,8 @@ int main() // write header cmocka_unit_test_setup_teardown(when_zfpWriteHeaderMagic_expect_numBitsWrittenEqualToZFP_MAGIC_BITS, setup, teardown), + cmocka_unit_test_setup_teardown(when_zfpFieldMetadataCalled_onInvalidSize_expect_ZFP_META_NULL, setup, teardown), + cmocka_unit_test_setup_teardown(when_zfpFieldSetMetadataCalled_forInvalidMeta_expect_false, setup, teardown), cmocka_unit_test_setup_teardown(when_zfpWriteHeaderMagic_expect_24BitsAreCharsZfpFollowedBy8BitsZfpCodecVersion, setup, teardown), cmocka_unit_test_setup_teardown(when_zfpWriteHeaderMetadata_expect_numBitsWrittenEqualToZFP_META_BITS, setup, teardown), From 5555333ae24bcf05d02ae3dc2dcc8229bca8c62e Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 9 Nov 2021 16:07:00 -0800 Subject: [PATCH 011/277] add promote/demote util tests --- tests/src/misc/CMakeLists.txt | 4 ++ tests/src/misc/testZfpPromote.c | 120 ++++++++++++++++++++++++++++++++ 2 files changed, 124 insertions(+) create mode 100644 tests/src/misc/testZfpPromote.c diff --git a/tests/src/misc/CMakeLists.txt b/tests/src/misc/CMakeLists.txt index 593a17ad8..d659bab60 100644 --- a/tests/src/misc/CMakeLists.txt +++ b/tests/src/misc/CMakeLists.txt @@ -6,6 +6,10 @@ add_executable(testZfpStream testZfpStream.c) target_link_libraries(testZfpStream cmocka zfp) add_test(NAME testZfpStream COMMAND testZfpStream) +add_executable(testZfpPromote testZfpPromote.c) +target_link_libraries(testZfpPromote cmocka zfp) +add_test(NAME testZfpPromote COMMAND testZfpPromote) + if(HAVE_LIBM_MATH) target_link_libraries(testZfpHeader m) target_link_libraries(testZfpStream m) diff --git a/tests/src/misc/testZfpPromote.c b/tests/src/misc/testZfpPromote.c new file mode 100644 index 000000000..567580e8d --- /dev/null +++ b/tests/src/misc/testZfpPromote.c @@ -0,0 +1,120 @@ +#include "zfp.h" + +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +static void +given_int8_when_promoteToInt32_expect_demoteToInt8Matches(void **state) +{ + uint dims = 3; + uint sz = 1u << (2 * dims); + int8* iblock8 = (int8*)malloc(sizeof(int8)*sz); + int8* oblock8 = (int8*)calloc(sz, sizeof(int8)); + int32* block32 = (int32*)malloc(sizeof(int32)*sz); + + assert_non_null(iblock8); + assert_non_null(oblock8); + assert_non_null(block32); + + int8 i; + for (i = 0; i < sz; i++) + iblock8[i] = i; + + zfp_promote_int8_to_int32(block32, iblock8, dims); + zfp_demote_int32_to_int8(oblock8, block32, dims); + + for (i = 0; i < sz; i++) + assert_int_equal(iblock8[i], oblock8[i]); +} + +static void +given_uint8_when_promoteToInt32_expect_demoteToUInt8Matches(void **state) +{ + uint dims = 3; + uint sz = 1u << (2 * dims); + uint8* iblock8 = (uint8*)malloc(sizeof(uint8)*sz); + uint8* oblock8 = (uint8*)calloc(sz, sizeof(uint8)); + int32* block32 = (int32*)malloc(sizeof(int32)*sz); + + assert_non_null(iblock8); + assert_non_null(oblock8); + assert_non_null(block32); + + uint8 i; + for (i = 0; i < sz; i++) + iblock8[i] = i; + + zfp_promote_uint8_to_int32(block32, iblock8, dims); + zfp_demote_int32_to_uint8(oblock8, block32, dims); + + for (i = 0; i < sz; i++) + assert_int_equal(iblock8[i], oblock8[i]); +} + +static void +given_int16_when_promoteToInt32_expect_demoteToInt16Matches(void **state) +{ + uint dims = 3; + uint sz = 1u << (2 * dims); + int16* iblock16 = (int16*)malloc(sizeof(int16)*sz); + int16* oblock16 = (int16*)calloc(sz, sizeof(int16)); + int32* block32 = (int32*)malloc(sizeof(int32)*sz); + + assert_non_null(iblock16); + assert_non_null(oblock16); + assert_non_null(block32); + + int16 i; + for (i = 0; i < sz; i++) + iblock16[i] = i; + + zfp_promote_int16_to_int32(block32, iblock16, dims); + zfp_demote_int32_to_int16(oblock16, block32, dims); + + for (i = 0; i < sz; i++) + assert_int_equal(iblock16[i], oblock16[i]); +} + +static void +given_uint16_when_promoteToInt32_expect_demoteToUInt16Matches(void **state) +{ + uint dims = 3; + uint sz = 1u << (2 * dims); + uint16* iblock16 = (uint16*)malloc(sizeof(uint16)*sz); + uint16* oblock16 = (uint16*)calloc(sz, sizeof(uint16)); + int32* block32 = (int32*)malloc(sizeof(int32)*sz); + + assert_non_null(iblock16); + assert_non_null(oblock16); + assert_non_null(block32); + + uint16 i; + for (i = 0; i < sz; i++) + iblock16[i] = i; + + zfp_promote_uint16_to_int32(block32, iblock16, dims); + zfp_demote_int32_to_uint16(oblock16, block32, dims); + + for (i = 0; i < sz; i++) + assert_int_equal(iblock16[i], oblock16[i]); +} + +int main() +{ + const struct CMUnitTest tests[] = { + cmocka_unit_test(given_int8_when_promoteToInt32_expect_demoteToInt8Matches), + cmocka_unit_test(given_uint8_when_promoteToInt32_expect_demoteToUInt8Matches), + cmocka_unit_test(given_int16_when_promoteToInt32_expect_demoteToInt16Matches), + cmocka_unit_test(given_uint16_when_promoteToInt32_expect_demoteToUInt16Matches), + }; + + return cmocka_run_group_tests(tests, NULL, NULL); +} From 142a11f723cf81c2e9a42840da825bdf74bde191 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Wed, 10 Nov 2021 16:35:24 -0800 Subject: [PATCH 012/277] Setup github actions config --- .github/workflows/main.yml | 84 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 .github/workflows/main.yml diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml new file mode 100644 index 000000000..c414debcc --- /dev/null +++ b/.github/workflows/main.yml @@ -0,0 +1,84 @@ +name: run tests +on: push +env: + BUILD_TYPE: Release +jobs: + build: + runs-on: ${{matrix.os}} + strategy: + matrix: + include: + - os: ubuntu-latest + cxx_compiler: g++-10 + c_compiler: gcc-10 + omp: ON + target: all + + - os: ubuntu-latest + cxx_compiler: clang++ + c_compiler: clang + omp: ON + target: all + + - os: windows-latest + cxx_compiler: msbuild + c_compiler: msbuild + omp: ON + target: ALL_BUILD + + #- os: windows-latest + # cxx_compiler: x86_64-w64-mingw32-g++ + # c_compiler: x86_64-w64-mingw32-gcc + # omp: ON + # target: all + # generator: '-G "MinGW Makefiles"' + + - os: macos-latest + cxx_compiler: clang++ + c_compiler: clang + omp: OFF + target: all + + - os: macos-latest + cxx_compiler: g++-10 + c_compiler: gcc-10 + omp: ON + target: all + steps: + - uses: actions/checkout@v2 + + - name: Setup MSBuild (Windows) + id: msbuild + if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} + uses: microsoft/setup-msbuild@v1.0.3 + + #- name: Setup MinGW (Windows) + # id: mingw + # if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'x86_64-w64-mingw32-g++'}} + # uses: egor-tensin/setup-mingw@v2 + + - name: CI Settings + id: settings + run: | + echo "os: ${{matrix.os}}" + echo "compilers:" + echo " cxx: ${{matrix.cxx_compiler}}" + echo " c: ${{matrix.c_compiler}}" + echo "OpenMP: ${{matrix.omp}}" + + - name: Run CMake + id: cmake + run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=${{matrix.omp}} + + - name: Build + id: build + run: cmake --build ${{github.workspace}}/build --target ${{matrix.target}} --config ${{env.BUILD_TYPE}} + + - name: Run Tests + id: test + working-directory: ${{github.workspace}}/build + run: ctest -C ${{env.BUILD_TYPE}} + + # Interactive Debug -> see: https://github.com/mxschmitt/action-tmate + #- name: Setup Debug Session + # uses: mxschmitt/action-tmate@v3 From 8441d706340bd09ec19364805b3aaf8395935f89 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 10:20:06 -0800 Subject: [PATCH 013/277] Fix inconsistent coding style in zfpcodec --- array/zfpcodec.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/array/zfpcodec.h b/array/zfpcodec.h index 18cbe1f70..e7e9bc279 100644 --- a/array/zfpcodec.h +++ b/array/zfpcodec.h @@ -155,7 +155,7 @@ class zfp1 : public zfp_base<1, Scalar> { size_t encode_block(size_t offset, uint shape, const Scalar* block) const { return shape ? encode_block_strided(offset, shape, block, 1) - : zfp_base<1, Scalar>::encode_block(offset, block); + : encode_block(offset, block); } // encode 1D block from strided storage From 05156f50d66f6c835243af8a2deeb0a7bfd399f3 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 10:23:16 -0800 Subject: [PATCH 014/277] Replace ieeecodec with generic codec templated on scalar type --- CHANGELOG.md | 2 + array/gencodec.h | 428 ++++++++++++++++++++++++++++++++++++++++++ array/ieeecodec.h | 260 ------------------------- array/zfp/genheader.h | 76 ++++++++ docs/source/codec.inc | 53 ++++-- 5 files changed, 548 insertions(+), 271 deletions(-) create mode 100644 array/gencodec.h delete mode 100644 array/ieeecodec.h create mode 100644 array/zfp/genheader.h diff --git a/CHANGELOG.md b/CHANGELOG.md index e37c1c015..fdf470a2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,8 @@ few of the API changes, other than to cfp, should impact existing code. - `zfp::array::size_bytes()` allows querying the size of different components of an array object (e.g., payload, cache, index, metadata, ...). - Templated C++ wrappers around the low-level C API. +- A generic codec for storing blocks of uncompressed scalars in zfp's + C++ arrays. - Additional functions for querying `zfp_field` and `zfp_stream` structs. - `zfp_config`: struct that encapsulates compression mode and parameters. - Rounding modes for reducing bias in compression errors. diff --git a/array/gencodec.h b/array/gencodec.h new file mode 100644 index 000000000..ebb727fa0 --- /dev/null +++ b/array/gencodec.h @@ -0,0 +1,428 @@ +#ifndef ZFP_GENERIC_CODEC_H +#define ZFP_GENERIC_CODEC_H + +// This CODEC allows interfacing with the zfp::array classes via a user-facing +// scalar type, ExternalType (e.g., double), while storing data in memory using +// a possibly less precise scalar type, InternalType (e.g., float). Using +// zfp's caching mechanism, blocks of data may reside for some time in cache +// as ExternalType. This potentially allows a sequence of more precise +// operations to be performed on the data before it is down-converted to +// InternalType and stored to memory. When ExternalType = InternalType, this +// CODEC allows defining arrays that support the full zfp array API but use +// uncompressed storage. To use this CODEC, pass it as the Codec template +// parameter to a zfp::array class of matching dimensionality. + +#include +#include +#include +#include "zfp.h" +#include "zfparray.h" +#include "zfp/memory.h" +#include "zfp/traits.h" + +namespace zfp { +namespace codec { + +// abstract base class for storing 1D-4D uncompressed blocks of scalars +template < + uint dims, // data dimensionality (1-4) + typename ExternalType, // scalar type exposed through array API + typename InternalType = ExternalType // scalar type used for storage +> +class generic_base { +protected: + // default constructor + generic_base() : + bytes(0), + buffer(0) + {} + +public: + // conservative buffer size for current codec settings + size_t buffer_size(const zfp_field* field) const + { + // empty field case + if (!field->nx && !field->ny && !field->nz && !field->nw) + return 0; + // count number of blocks spanned by field + size_t bx = (std::max(field->nx, size_t(1)) + 3) / 4; + size_t by = (std::max(field->ny, size_t(1)) + 3) / 4; + size_t bz = (std::max(field->nz, size_t(1)) + 3) / 4; + size_t bw = (std::max(field->nw, size_t(1)) + 3) / 4; + size_t blocks = bx * by * bz * bw; + return blocks * block_size * sizeof(InternalType); + } + + // open + void open(void* data, size_t size) + { + bytes = size; + buffer = static_cast(data); + } + + // close bit stream + void close() + { + bytes = 0; + buffer = 0; + } + + // pointer to beginning of bit stream + void* data() const { return static_cast(buffer); } + + // compression mode + zfp_mode mode() const { return zfp_mode_fixed_rate; } + + // rate in compressed bits/value (equals precision) + double rate() const { return static_cast(precision()); } + + // precision in uncompressed bits/value + uint precision() const { return internal_size_bits; } + + // accuracy as absolute error tolerance (unsupported) + double accuracy() const { return -1; } + + // compression parameters (all compression modes) + void params(uint* minbits, uint* maxbits, uint* maxprec, int* minexp) const + { + if (minbits) + *minbits = block_size_bits; + if (maxbits) + *maxbits = block_size_bits; + if (maxprec) + *maxprec = precision(); + if (minexp) + *minexp = ZFP_MIN_EXP; + } + + // enable reversible (lossless) mode + void set_reversible() + { + throw zfp::exception("zfp generic codec does not support reversible mode"); + } + + // set rate in compressed bits/value (equals precision) + double set_rate(double rate, bool) + { + return static_cast(set_precision(static_cast(rate))); + } + + // set precision in uncompressed bits/value (must equal InternalType width) + uint set_precision(uint precision) + { + if (precision != internal_size_bits) + throw zfp::exception("zfp generic codec precision mismatch"); + return precision; + } + + // set accuracy as absolute error tolerance + double set_accuracy(double) + { + throw zfp::exception("zfp generic codec does not support fixed-accuracy mode"); + return -1; + } + + // set expert mode parameters + bool set_params(uint, uint, uint, int) + { + throw zfp::exception("zfp generic codec does not support expert mode"); + return false; + } + + // byte size of codec data structure components indicated by mask + size_t size_bytes(uint mask = ZFP_DATA_ALL) const + { + size_t size = 0; + if (mask & ZFP_DATA_META) + size += sizeof(*this); + return size; + } + + // unit of allocated data in bytes + static size_t alignment() { return sizeof(InternalType); } + + static const zfp_type type = zfp::trait::type; // scalar type + + // zfp::codec::generic_base::header class for array (de)serialization + #include "zfp/genheader.h" + +protected: + // pointer to beginning of block + InternalType* begin(size_t offset) const + { + if (offset % internal_size_bits) + throw zfp::exception("zfp generic codec bit offset alignment error"); + return buffer + offset / internal_size_bits; + } + + // store full contiguous block to memory + size_t encode_block(size_t offset, const ExternalType* block) const + { + InternalType* ptr = begin(offset); + for (size_t n = block_size; n--;) + *ptr++ = static_cast(*block++); + return block_size_bits; + } + + // load full contiguous block from memory + size_t decode_block(size_t offset, ExternalType* block) const + { + const InternalType* ptr = begin(offset); + for (size_t n = block_size; n--;) + *block++ = static_cast(*ptr++); + return block_size_bits; + } + + // constants associated with template arguments + static const size_t internal_size_bits = sizeof(InternalType) * CHAR_BIT; + static const size_t block_size = 1u << (2 * dims); + static const size_t block_size_bits = block_size * internal_size_bits; + + size_t bytes; // number of bytes of storage + InternalType* buffer; // pointer to storage managed by block store +}; + +// 1D codec +template +class generic1 : public generic_base<1, ExternalType, InternalType> { +public: + // encode contiguous 1D block + size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + { + return shape ? encode_block_strided(offset, shape, block, 1) + : encode_block(offset, block); + } + + // encode 1D block from strided storage + size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx) const + { + InternalType* q = begin(offset); + size_t nx = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + } + for (size_t x = 0; x < nx; x++, p += sx, q++) + *q = static_cast(*p); + return block_size_bits; + } + + // decode contiguous 1D block + size_t decode_block(size_t offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1) + : decode_block(offset, block); + } + + // decode 1D block to strided storage + size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx) const + { + const InternalType* q = begin(offset); + size_t nx = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + } + for (size_t x = 0; x < nx; x++, p += sx, q++) + *p = static_cast(*q); + return block_size_bits; + } + +protected: + using generic_base<1, ExternalType, InternalType>::begin; + using generic_base<1, ExternalType, InternalType>::encode_block; + using generic_base<1, ExternalType, InternalType>::decode_block; + using generic_base<1, ExternalType, InternalType>::block_size_bits; +}; + +// 2D codec +template +class generic2 : public generic_base<2, ExternalType, InternalType> { +public: + // encode contiguous 2D block + size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + { + return shape ? encode_block_strided(offset, shape, block, 1, 4) + : encode_block(offset, block); + } + + // encode 2D block from strided storage + size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const + { + InternalType* q = begin(offset); + size_t nx = 4; + size_t ny = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + ny -= shape & 3u; shape >>= 2; + } + for (size_t y = 0; y < ny; y++, p += sy - (ptrdiff_t)nx * sx, q += 4 - nx) + for (size_t x = 0; x < nx; x++, p += sx, q++) + *q = static_cast(*p); + return block_size_bits; + } + + // decode contiguous 2D block + size_t decode_block(size_t offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4) + : decode_block(offset, block); + } + + // decode 2D block to strided storage + size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const + { + const InternalType* q = begin(offset); + size_t nx = 4; + size_t ny = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + ny -= shape & 3u; shape >>= 2; + } + for (size_t y = 0; y < ny; y++, p += sy - (ptrdiff_t)nx * sx, q += 4 - nx) + for (size_t x = 0; x < nx; x++, p += sx, q++) + *p = static_cast(*q); + return block_size_bits; + } + +protected: + using generic_base<2, ExternalType, InternalType>::begin; + using generic_base<2, ExternalType, InternalType>::encode_block; + using generic_base<2, ExternalType, InternalType>::decode_block; + using generic_base<2, ExternalType, InternalType>::block_size_bits; +}; + +// 3D codec +template +class generic3 : public generic_base<3, ExternalType, InternalType> { +public: + // encode contiguous 3D block + size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + { + return shape ? encode_block_strided(offset, shape, block, 1, 4, 16) + : encode_block(offset, block); + } + + // encode 3D block from strided storage + size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + { + InternalType* q = begin(offset); + size_t nx = 4; + size_t ny = 4; + size_t nz = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + ny -= shape & 3u; shape >>= 2; + nz -= shape & 3u; shape >>= 2; + } + for (size_t z = 0; z < nz; z++, p += sz - (ptrdiff_t)ny * sy, q += 16 - 4 * ny) + for (size_t y = 0; y < ny; y++, p += sy - (ptrdiff_t)nx * sx, q += 4 - nx) + for (size_t x = 0; x < nx; x++, p += sx, q++) + *q = static_cast(*p); + return block_size_bits; + } + + // decode contiguous 3D block + size_t decode_block(size_t offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) + : decode_block(offset, block); + } + + // decode 3D block to strided storage + size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + { + const InternalType* q = begin(offset); + size_t nx = 4; + size_t ny = 4; + size_t nz = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + ny -= shape & 3u; shape >>= 2; + nz -= shape & 3u; shape >>= 2; + } + for (size_t z = 0; z < nz; z++, p += sz - (ptrdiff_t)ny * sy, q += 16 - 4 * ny) + for (size_t y = 0; y < ny; y++, p += sy - (ptrdiff_t)nx * sx, q += 4 - nx) + for (size_t x = 0; x < nx; x++, p += sx, q++) + *p = static_cast(*q); + return block_size_bits; + } + +protected: + using generic_base<3, ExternalType, InternalType>::begin; + using generic_base<3, ExternalType, InternalType>::encode_block; + using generic_base<3, ExternalType, InternalType>::decode_block; + using generic_base<3, ExternalType, InternalType>::block_size_bits; +}; + +// 4D codec +template +class generic4 : public generic_base<4, ExternalType, InternalType> { +public: + // encode contiguous 4D block + size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + { + return shape ? encode_block_strided(offset, shape, block, 1, 4, 16, 64) + : encode_block(offset, block); + } + + // encode 4D block from strided storage + size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + { + InternalType* q = begin(offset); + size_t nx = 4; + size_t ny = 4; + size_t nz = 4; + size_t nw = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + ny -= shape & 3u; shape >>= 2; + nz -= shape & 3u; shape >>= 2; + nw -= shape & 3u; shape >>= 2; + } + for (size_t w = 0; w < nw; w++, p += sw - (ptrdiff_t)nz * sz, q += 64 - 16 * nz) + for (size_t z = 0; z < nz; z++, p += sz - (ptrdiff_t)ny * sy, q += 16 - 4 * ny) + for (size_t y = 0; y < ny; y++, p += sy - (ptrdiff_t)nx * sx, q += 4 - nx) + for (size_t x = 0; x < nx; x++, p += sx, q++) + *q = static_cast(*p); + return block_size_bits; + } + + // decode contiguous 4D block + size_t decode_block(size_t offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) + : decode_block(offset, block); + } + + // decode 4D block to strided storage + size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + { + const InternalType* q = begin(offset); + size_t nx = 4; + size_t ny = 4; + size_t nz = 4; + size_t nw = 4; + if (shape) { + nx -= shape & 3u; shape >>= 2; + ny -= shape & 3u; shape >>= 2; + nz -= shape & 3u; shape >>= 2; + nw -= shape & 3u; shape >>= 2; + } + for (size_t w = 0; w < nw; w++, p += sw - (ptrdiff_t)nz * sz, q += 64 - 16 * nz) + for (size_t z = 0; z < nz; z++, p += sz - (ptrdiff_t)ny * sy, q += 16 - 4 * ny) + for (size_t y = 0; y < ny; y++, p += sy - (ptrdiff_t)nx * sx, q += 4 - nx) + for (size_t x = 0; x < nx; x++, p += sx, q++) + *p = static_cast(*q); + return block_size_bits; + } + +protected: + using generic_base<4, ExternalType, InternalType>::begin; + using generic_base<4, ExternalType, InternalType>::encode_block; + using generic_base<4, ExternalType, InternalType>::decode_block; + using generic_base<4, ExternalType, InternalType>::block_size_bits; +}; + +} // codec +} // zfp + +#endif diff --git a/array/ieeecodec.h b/array/ieeecodec.h deleted file mode 100644 index be2626889..000000000 --- a/array/ieeecodec.h +++ /dev/null @@ -1,260 +0,0 @@ -#ifndef ZFP_IEEE_CODEC_H -#define ZFP_IEEE_CODEC_H - -#include -#include -#include -#include "zfp.h" -#include "zfpcpp.h" -#include "zfp/traits.h" - -namespace zfp { - -// base class for IEEE-754 coding of {float, double} x {1D, 2D, 3D} data -template -class ieee_codec_base { -protected: - // constructor takes pre-allocated buffer of compressed blocks - ieee_codec_base(void* data, size_t size) : - data(data), - size(size) - {} - -public: - // destructor - ~ieee_codec_base() - {} - - // return nearest rate supported - static double nearest_rate(double target_rate) - { - size_t block_bits = static_cast(target_rate * block_size); - size_t word_bits = stream_alignment(); - size_t words = std::max((block_bits + word_bits - 1) / word_bits, size_t(1)); - return static_cast(words * word_bits) / block_size; - } - - // rate in bits/value - double rate() const { return double(zfp->maxbits) / block_size; } - - // set rate in bits/value - double set_rate(double rate) { return zfp_stream_set_rate(zfp, rate, type, dims, zfp_true); } - - static const zfp_type type = zfp::trait::type; // scalar type - - // zfp::ieee_codec_base::header class for array (de)serialization - #include "zfp/ieeeheader.h" - -protected: - // encode full contiguous block - size_t encode_block(size_t offset, const Scalar* block) - { - ptrdiff_t = offset / sizeof(); - // copy here and optionally convert - return block_size * rate; - - stream_wseek(zfp->stream, offset); - size_t size = zfp::encode_block(zfp, block); - size += zfp_stream_flush(zfp); - return size; - } - - // decode full contiguous block - size_t decode_block(size_t offset, Scalar* block) - { - stream_rseek(zfp->stream, offset); - size_t size = zfp::decode_block(zfp, block); - size += zfp_stream_align(zfp); - return size; - } - - static const size_t block_size = 1u << (2 * dims); // block size in number of scalars - - void* data; - size_t size; -}; - -// zfp codec templated on scalar type and number of dimensions -template -class zfp_codec; - -// 1D codec -template -class zfp_codec : public zfp_codec_base { -public: - // constructor takes pre-allocated buffer of compressed blocks - zfp_codec(void* data, size_t size) : zfp_codec_base(data, size) {} - - // encode contiguous 1D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) - { - return shape ? encode_block_strided(offset, shape, block, 1) - : zfp_codec_base::encode_block(offset, block); - } - - // encode 1D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx) - { - size_t size; - stream_wseek(zfp->stream, offset); - if (shape) { - uint nx = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(zfp, p, nx, sx); - } - else - size = zfp::encode_block_strided(zfp, p, sx); - size += zfp_stream_flush(zfp); - return size; - } - - // decode contiguous 1D block - size_t decode_block(size_t offset, uint shape, Scalar* block) - { - return shape ? decode_block_strided(offset, shape, block, 1) - : decode_block(offset, block); - } - - // decode 1D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx) - { - size_t size; - stream_rseek(zfp->stream, offset); - if (shape) { - uint nx = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(zfp, p, nx, sx); - } - else - size = zfp::decode_block_strided(zfp, p, sx); - size += zfp_stream_align(zfp); - return size; - } - -protected: - using zfp_codec_base::encode_block; - using zfp_codec_base::decode_block; - using zfp_codec_base::zfp; -}; - -// 2D codec -template -class zfp_codec : public zfp_codec_base { -public: - // constructor takes pre-allocated buffer of compressed blocks - zfp_codec(void* data, size_t size) : zfp_codec_base(data, size) {} - - // encode contiguous 2D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) - { - return shape ? encode_block_strided(offset, shape, block, 1, 4) - : encode_block(offset, block); - } - - // encode 2D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) - { - size_t size; - stream_wseek(zfp->stream, offset); - if (shape) { - uint nx = 4 - (shape & 3u); shape >>= 2; - uint ny = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(zfp, p, nx, ny, sx, sy); - } - else - size = zfp::encode_block_strided(zfp, p, sx, sy); - size += zfp_stream_flush(zfp); - return size; - } - - // decode contiguous 2D block - size_t decode_block(size_t offset, uint shape, Scalar* block) - { - return shape ? decode_block_strided(offset, shape, block, 1, 4) - : decode_block(offset, block); - } - - // decode 2D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) - { - size_t size; - stream_rseek(zfp->stream, offset); - if (shape) { - uint nx = 4 - (shape & 3u); shape >>= 2; - uint ny = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(zfp, p, nx, ny, sx, sy); - } - else - size = zfp::decode_block_strided(zfp, p, sx, sy); - size += zfp_stream_align(zfp); - return size; - } - -protected: - using zfp_codec_base::encode_block; - using zfp_codec_base::decode_block; - using zfp_codec_base::zfp; -}; - -// 3D codec -template -class zfp_codec : public zfp_codec_base { -public: - // constructor takes pre-allocated buffer of compressed blocks - zfp_codec(void* data, size_t size) : zfp_codec_base(data, size) {} - - // encode contiguous 3D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) - { - return shape ? encode_block_strided(offset, shape, block, 1, 4, 16) - : encode_block(offset, block); - } - - // encode 3D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) - { - size_t size; - stream_wseek(zfp->stream, offset); - if (shape) { - uint nx = 4 - (shape & 3u); shape >>= 2; - uint ny = 4 - (shape & 3u); shape >>= 2; - uint nz = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(zfp, p, nx, ny, nz, sx, sy, sz); - } - else - size = zfp::encode_block_strided(zfp, p, sx, sy, sz); - size += zfp_stream_flush(zfp); - return size; - } - - // decode contiguous 3D block - size_t decode_block(size_t offset, uint shape, Scalar* block) - { - return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) - : decode_block(offset, block); - } - - // decode 3D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) - { - size_t size; - stream_rseek(zfp->stream, offset); - if (shape) { - uint nx = 4 - (shape & 3u); shape >>= 2; - uint ny = 4 - (shape & 3u); shape >>= 2; - uint nz = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(zfp, p, nx, ny, nz, sx, sy, sz); - } - else - size = zfp::decode_block_strided(zfp, p, sx, sy, sz); - size += zfp_stream_align(zfp); - return size; - } - -protected: - using zfp_codec_base::encode_block; - using zfp_codec_base::decode_block; - using zfp_codec_base::zfp; -}; - -} - -#endif diff --git a/array/zfp/genheader.h b/array/zfp/genheader.h new file mode 100644 index 000000000..8beec0880 --- /dev/null +++ b/array/zfp/genheader.h @@ -0,0 +1,76 @@ +// zfp::codec::generic_base::header +class header : public zfp::array::header { +public: + // serialization: construct header from array + header(const zfp::array& a) : + zfp::array::header(a), + bit_rate(static_cast(a.rate())) + { + buffer[0] = magic; + buffer[1] = 0; // TODO: codec identifier (dimensionality, internal type) + buffer[2] = static_cast(bit_rate); + buffer[3] = static_cast(type); + buffer[4] = static_cast(nx); + buffer[5] = static_cast(ny); + buffer[6] = static_cast(nz); + buffer[7] = static_cast(nw); + } + + // deserialization: construct header from memory buffer of optional size + header(const void* data, size_t bytes = 0) : + bit_rate(0) + { + // ensure byte size matches + if (bytes && bytes != byte_size) + throw zfp::exception("zfp generic header length does not match expectations"); + else { + // copy and parse header + std::memcpy(buffer, data, byte_size); + if (buffer[0] != magic) + throw zfp::exception("zfp generic header is corrupt"); + bit_rate = static_cast(buffer[2]); + type = static_cast(buffer[3]); + nx = static_cast(buffer[4]); + ny = static_cast(buffer[5]); + nz = static_cast(buffer[6]); + nw = static_cast(buffer[7]); + } + } + + virtual ~header() {} + + // rate in bits per value + double rate() const { return static_cast(bit_rate); } + + // header data + const void* data() const { return buffer; } + + // header byte size + size_t size_bytes(uint mask = ZFP_DATA_HEADER) const + { + size_t size = 0; + if (mask & ZFP_DATA_META) + size += sizeof(*this) - byte_size; + if (mask & ZFP_DATA_HEADER) + size += byte_size; + return size; + } + +protected: + // magic word + static const uint64 magic = UINT64C(0x000000008570667a); + + // header size measured in bits, bytes, and 64-bit words + static const size_t word_size = 8; + static const size_t byte_size = word_size * sizeof(uint64); + static const size_t bit_size = byte_size * CHAR_BIT; + + using zfp::array::header::type; + using zfp::array::header::nx; + using zfp::array::header::ny; + using zfp::array::header::nz; + using zfp::array::header::nw; + + size_t bit_rate; // array rate in bits per value + uint64 buffer[word_size]; // header data +}; diff --git a/docs/source/codec.inc b/docs/source/codec.inc index be9c7b252..5b4c0b67b 100644 --- a/docs/source/codec.inc +++ b/docs/source/codec.inc @@ -24,6 +24,14 @@ codec does not support a certain compression mode, it should throw an Codecs reside in the :code:`zfp::codec` namespace, e.g., :code:`zfp::codec::zfp3` is the default codec for 3D arrays. +As of |zfp| |cpprelease|, there is in addition to the default |zfp| codec +a "generic" codec that allows storing data in |zfp| arrays in "uncompressed" +form using any scalar type (specified as a template parameter). This +"internal" scalar type may differ from the "external" scalar type exposed +to the user through the :cpp:class:`zfp::array` API. For instance, the +internal type may be :code:`float` while the external type is :code:`double`, +which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. + .. cpp:namespace:: zfp::codec .. cpp:class:: codec @@ -164,28 +172,51 @@ Codecs reside in the :code:`zfp::codec` namespace, e.g., ---- -.. cpp:function:: size_t codec::encode_block(size_t offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec::encode_block(size_t offset, const Scalar* block) const + + Encode contiguous *block* of |4powd| scalars and store at specified bit + *offset* within compressed-data buffer. Return the number of bits of + compressed storage for the block, excluding any necessary padding. This + method must flush any buffered compressed data without counting any padding + (e.g., for byte alignment) in the compressed size (unless the codec requires + alignment of the bit offsets). + +---- + +.. cpp:function:: size_t codec::decode_block(size_t offset, Scalar* block) const + + Decode contiguous *block* of |4powd| scalars from specified bit *offset* + within compressed-data buffer (see :cpp:func:`codec::encode_block`). + Return number of bits of compressed data decoded, excluding any padding + bits, i.e., the same value reported in encoding. + +---- + +.. cpp:function:: size_t codec1::encode_block(size_t offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec2::encode_block(size_t offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec3::encode_block(size_t offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec4::encode_block(size_t offset, uint shape, const Scalar* block) const Encode contiguous *block* of data of given *shape* and store at specified bit *offset* within compressed-data buffer. Return the number of bits of - compressed storage for the block. + compressed storage for the block (see also :cpp:func:`codec::encode_block`). The *shape* is a (2 |times| *d*)-bit encoding of the size of the *d*-dimensional block. For each successive pair of bits *s* of *shape*, the block size in the corresponding dimension is *n* = 4 - *s*, where 0 |leq| *s* |leq| 3. Thus, *shape* = 0 implies a full block of |4powd| values. The size of the fastest varying dimension is specified in the - least significant bits of *shape*. These methods must flush any buffered - compressed data without counting any padding (e.g., for byte alignment) - in the compressed size (unless the codec requires alignment of the bit - offsets). + least significant bits of *shape*. ---- -.. cpp:function:: size_t codec::decode_block(size_t offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec1::decode_block(size_t offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec2::decode_block(size_t offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec3::decode_block(size_t offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec4::decode_block(size_t offset, uint shape, Scalar* block) const Decode contiguous *block* of data of given *shape* from specified bit - *offset* within compressed-data buffer (see + *offset* within compressed-data buffer (see also :cpp:func:`codec1::encode_block`). Return number of bits of compressed data decoded, excluding any padding bits, i.e., the same value reported in encoding. @@ -200,7 +231,7 @@ Codecs reside in the :code:`zfp::codec` namespace, e.g., Encode block of data stored at *p* with strides *sx*, *sy*, *sz*, and *sw*. See :c:type:`zfp_field` for information on strided storage. The *shape*, *offset*, and return value are as in - :cpp:func:`codec::encode_block`. + :cpp:func:`codec1::encode_block`. ---- @@ -210,6 +241,6 @@ Codecs reside in the :code:`zfp::codec` namespace, e.g., .. cpp:function:: size_t codec4::decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const Decode block to strided storage pointed to by *p* with strides *sx*, *sy*, - *sz*, and *sw. See :c:type:`zfp_field` for information on strided storage. + *sz*, and *sw*. See :c:type:`zfp_field` for information on strided storage. The *shape*, *offset*, and return value are as in - :cpp:func:`codec::decode_block`. + :cpp:func:`codec1::decode_block`. From 662fc0f42694e2a395d3e20b20c5823ac6b2075f Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 10:43:02 -0800 Subject: [PATCH 015/277] Add size_bytes() to cfp --- cfp/include/cfparray1d.h | 1 + cfp/include/cfparray1f.h | 1 + cfp/include/cfparray2d.h | 1 + cfp/include/cfparray2f.h | 1 + cfp/include/cfparray3d.h | 1 + cfp/include/cfparray3f.h | 1 + cfp/include/cfparray4d.h | 1 + cfp/include/cfparray4f.h | 1 + cfp/src/cfparray.cpp | 8 ++++++++ cfp/src/template/cfparray.cpp | 6 ++++++ docs/source/cfp.rst | 6 ++++++ 11 files changed, 28 insertions(+) diff --git a/cfp/include/cfparray1d.h b/cfp/include/cfparray1d.h index 193f6d09b..155c9d67a 100644 --- a/cfp/include/cfparray1d.h +++ b/cfp/include/cfparray1d.h @@ -110,6 +110,7 @@ typedef struct { void (*set_cache_size)(cfp_array1d self, size_t bytes); void (*clear_cache)(const cfp_array1d self); void (*flush_cache)(const cfp_array1d self); + size_t (*size_bytes)(const cfp_array1d self, uint mask); size_t (*compressed_size)(const cfp_array1d self); void* (*compressed_data)(const cfp_array1d self); size_t (*size)(const cfp_array1d self); diff --git a/cfp/include/cfparray1f.h b/cfp/include/cfparray1f.h index ac48d65e9..e271a8c20 100644 --- a/cfp/include/cfparray1f.h +++ b/cfp/include/cfparray1f.h @@ -110,6 +110,7 @@ typedef struct { void (*set_cache_size)(cfp_array1f self, size_t bytes); void (*clear_cache)(const cfp_array1f self); void (*flush_cache)(const cfp_array1f self); + size_t (*size_bytes)(const cfp_array1f self, uint mask); size_t (*compressed_size)(const cfp_array1f self); void* (*compressed_data)(const cfp_array1f self); size_t (*size)(const cfp_array1f self); diff --git a/cfp/include/cfparray2d.h b/cfp/include/cfparray2d.h index 715d6a973..5acbc0c6b 100644 --- a/cfp/include/cfparray2d.h +++ b/cfp/include/cfparray2d.h @@ -111,6 +111,7 @@ typedef struct { void (*set_cache_size)(cfp_array2d self, size_t bytes); void (*clear_cache)(const cfp_array2d self); void (*flush_cache)(const cfp_array2d self); + size_t (*size_bytes)(const cfp_array2d self, uint mask); size_t (*compressed_size)(const cfp_array2d self); void* (*compressed_data)(const cfp_array2d self); size_t (*size)(const cfp_array2d self); diff --git a/cfp/include/cfparray2f.h b/cfp/include/cfparray2f.h index 1d7a23ce6..f55a1a656 100644 --- a/cfp/include/cfparray2f.h +++ b/cfp/include/cfparray2f.h @@ -111,6 +111,7 @@ typedef struct { void (*set_cache_size)(cfp_array2f self, size_t bytes); void (*clear_cache)(const cfp_array2f self); void (*flush_cache)(const cfp_array2f self); + size_t (*size_bytes)(const cfp_array2f self, uint mask); size_t (*compressed_size)(const cfp_array2f self); void* (*compressed_data)(const cfp_array2f self); size_t (*size)(const cfp_array2f self); diff --git a/cfp/include/cfparray3d.h b/cfp/include/cfparray3d.h index fa2b6f260..8a4b68fd8 100644 --- a/cfp/include/cfparray3d.h +++ b/cfp/include/cfparray3d.h @@ -112,6 +112,7 @@ typedef struct { void (*set_cache_size)(cfp_array3d self, size_t bytes); void (*clear_cache)(const cfp_array3d self); void (*flush_cache)(const cfp_array3d self); + size_t (*size_bytes)(const cfp_array3d self, uint mask); size_t (*compressed_size)(const cfp_array3d self); void* (*compressed_data)(const cfp_array3d self); size_t (*size)(const cfp_array3d self); diff --git a/cfp/include/cfparray3f.h b/cfp/include/cfparray3f.h index 779ea0c3f..b498b7678 100644 --- a/cfp/include/cfparray3f.h +++ b/cfp/include/cfparray3f.h @@ -112,6 +112,7 @@ typedef struct { void (*set_cache_size)(cfp_array3f self, size_t bytes); void (*clear_cache)(const cfp_array3f self); void (*flush_cache)(const cfp_array3f self); + size_t (*size_bytes)(const cfp_array3f self, uint mask); size_t (*compressed_size)(const cfp_array3f self); void* (*compressed_data)(const cfp_array3f self); size_t (*size)(const cfp_array3f self); diff --git a/cfp/include/cfparray4d.h b/cfp/include/cfparray4d.h index 098447a7f..81bb795ab 100644 --- a/cfp/include/cfparray4d.h +++ b/cfp/include/cfparray4d.h @@ -113,6 +113,7 @@ typedef struct { void (*set_cache_size)(cfp_array4d self, size_t bytes); void (*clear_cache)(const cfp_array4d self); void (*flush_cache)(const cfp_array4d self); + size_t (*size_bytes)(const cfp_array4d self, uint mask); size_t (*compressed_size)(const cfp_array4d self); void* (*compressed_data)(const cfp_array4d self); size_t (*size)(const cfp_array4d self); diff --git a/cfp/include/cfparray4f.h b/cfp/include/cfparray4f.h index efc6cf621..b13016153 100644 --- a/cfp/include/cfparray4f.h +++ b/cfp/include/cfparray4f.h @@ -113,6 +113,7 @@ typedef struct { void (*set_cache_size)(cfp_array4f self, size_t bytes); void (*clear_cache)(const cfp_array4f self); void (*flush_cache)(const cfp_array4f self); + size_t (*size_bytes)(const cfp_array4f self, uint mask); size_t (*compressed_size)(const cfp_array4f self); void* (*compressed_data)(const cfp_array4f self); size_t (*size)(const cfp_array4f self); diff --git a/cfp/src/cfparray.cpp b/cfp/src/cfparray.cpp index 9c4de0bba..358f480d7 100644 --- a/cfp/src/cfparray.cpp +++ b/cfp/src/cfparray.cpp @@ -27,6 +27,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array1f_set_cache_size, cfp_array1f_clear_cache, cfp_array1f_flush_cache, + cfp_array1f_size_bytes, cfp_array1f_compressed_size, cfp_array1f_compressed_data, cfp_array1f_size, @@ -129,6 +130,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array1d_set_cache_size, cfp_array1d_clear_cache, cfp_array1d_flush_cache, + cfp_array1d_size_bytes, cfp_array1d_compressed_size, cfp_array1d_compressed_data, cfp_array1d_size, @@ -231,6 +233,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array2f_set_cache_size, cfp_array2f_clear_cache, cfp_array2f_flush_cache, + cfp_array2f_size_bytes, cfp_array2f_compressed_size, cfp_array2f_compressed_data, cfp_array2f_size, @@ -336,6 +339,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array2d_set_cache_size, cfp_array2d_clear_cache, cfp_array2d_flush_cache, + cfp_array2d_size_bytes, cfp_array2d_compressed_size, cfp_array2d_compressed_data, cfp_array2d_size, @@ -441,6 +445,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array3f_set_cache_size, cfp_array3f_clear_cache, cfp_array3f_flush_cache, + cfp_array3f_size_bytes, cfp_array3f_compressed_size, cfp_array3f_compressed_data, cfp_array3f_size, @@ -548,6 +553,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array3d_set_cache_size, cfp_array3d_clear_cache, cfp_array3d_flush_cache, + cfp_array3d_size_bytes, cfp_array3d_compressed_size, cfp_array3d_compressed_data, cfp_array3d_size, @@ -655,6 +661,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array4f_set_cache_size, cfp_array4f_clear_cache, cfp_array4f_flush_cache, + cfp_array4f_size_bytes, cfp_array4f_compressed_size, cfp_array4f_compressed_data, cfp_array4f_size, @@ -764,6 +771,7 @@ const cfp_api CFP_NAMESPACE = { cfp_array4d_set_cache_size, cfp_array4d_clear_cache, cfp_array4d_flush_cache, + cfp_array4d_size_bytes, cfp_array4d_compressed_size, cfp_array4d_compressed_data, cfp_array4d_size, diff --git a/cfp/src/template/cfparray.cpp b/cfp/src/template/cfparray.cpp index b3b3e0760..70bb3c2dc 100644 --- a/cfp/src/template/cfparray.cpp +++ b/cfp/src/template/cfparray.cpp @@ -42,6 +42,12 @@ _t1(CFP_ARRAY_TYPE, set_rate)(CFP_ARRAY_TYPE self, double rate) return static_cast(self.object)->set_rate(rate); } +static size_t +_t1(CFP_ARRAY_TYPE, size_bytes)(CFP_ARRAY_TYPE self, uint mask) +{ + return static_cast(self.object)->size_bytes(mask); +} + static size_t _t1(CFP_ARRAY_TYPE, compressed_size)(CFP_ARRAY_TYPE self) { diff --git a/docs/source/cfp.rst b/docs/source/cfp.rst index 78465a573..6d94fdc75 100644 --- a/docs/source/cfp.rst +++ b/docs/source/cfp.rst @@ -346,6 +346,12 @@ not actually part of the |cfp| API. ---- +.. c:function:: size_t cfp.array.size_bytes(const cfp_array self, uint mask) + + See :cpp:func:`array::size_bytes`. + +---- + .. c:function:: size_t cfp.array.compressed_size(const cfp_array self) See :cpp:func:`array::compressed_size`. From 122918c4c7d90ebdf638d519f27bb21e811f6281 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 10:43:50 -0800 Subject: [PATCH 016/277] Fix incorrect loop counter types in simple.c --- examples/simple.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/simple.c b/examples/simple.c index bcddf4a74..d22613018 100644 --- a/examples/simple.c +++ b/examples/simple.c @@ -86,7 +86,7 @@ int main(int argc, char* argv[]) if (!decompress) { /* initialize array to be compressed */ - int i, j, k; + size_t i, j, k; for (k = 0; k < nz; k++) for (j = 0; j < ny; j++) for (i = 0; i < nx; i++) { From eb6cd2873dcf15352a6193f5887056de51414f22 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 10:51:41 -0800 Subject: [PATCH 017/277] Bug fix: reopen codec when compacting store --- array/zfp/store.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/array/zfp/store.h b/array/zfp/store.h index a06428de1..88e2f6363 100644 --- a/array/zfp/store.h +++ b/array/zfp/store.h @@ -115,8 +115,10 @@ class BlockStore { { size_t size = zfp::round_up(index.range(), codec.alignment() * CHAR_BIT) / CHAR_BIT; if (bytes > size) { + codec.close(); zfp::reallocate_aligned(data, size, ZFP_MEMORY_ALIGNMENT, bytes); bytes = size; + codec.open(data, bytes); } } From 323e3e7088f11a4df85cbd5784b8f9e286533b1d Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 14:07:44 -0800 Subject: [PATCH 018/277] Move Cache and Store to internal namespace --- array/zfp/cache.h | 4 +++- array/zfp/cache1.h | 8 +++++--- array/zfp/cache2.h | 8 +++++--- array/zfp/cache3.h | 8 +++++--- array/zfp/cache4.h | 8 +++++--- array/zfp/store.h | 4 +++- array/zfp/store1.h | 4 +++- array/zfp/store2.h | 4 +++- array/zfp/store3.h | 4 +++- array/zfp/store4.h | 4 +++- array/zfparray1.h | 4 ++-- array/zfparray2.h | 4 ++-- array/zfparray3.h | 4 ++-- array/zfparray4.h | 4 ++-- array/zfpcarray1.h | 4 ++-- array/zfpcarray2.h | 4 ++-- array/zfpcarray3.h | 4 ++-- array/zfpcarray4.h | 4 ++-- 18 files changed, 54 insertions(+), 34 deletions(-) diff --git a/array/zfp/cache.h b/array/zfp/cache.h index 91746e915..f1e415311 100644 --- a/array/zfp/cache.h +++ b/array/zfp/cache.h @@ -9,6 +9,7 @@ #endif namespace zfp { +namespace internal { // direct-mapped or two-way skew-associative write-back cache template @@ -274,6 +275,7 @@ class Cache { #endif }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/cache1.h b/array/zfp/cache1.h index d1c680211..98e1791fe 100644 --- a/array/zfp/cache1.h +++ b/array/zfp/cache1.h @@ -4,6 +4,7 @@ #include "cache.h" namespace zfp { +namespace internal { template class BlockCache1 { @@ -40,7 +41,7 @@ class BlockCache1 { // flush cache by compressing all modified cached blocks void flush() const { - for (typename zfp::Cache::const_iterator p = cache.first(); p; p++) { + for (typename zfp::internal::Cache::const_iterator p = cache.first(); p; p++) { if (p->tag.dirty()) { size_t block_index = p->tag.index() - 1; store.encode(block_index, p->line->data()); @@ -159,7 +160,7 @@ class BlockCache1 { { CacheLine* p = 0; size_t block_index = store.block_index(i); - typename zfp::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); + typename zfp::internal::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); size_t stored_block_index = tag.index() - 1; if (stored_block_index != block_index) { // write back occupied cache line if it is dirty @@ -194,6 +195,7 @@ class BlockCache1 { Store& store; // store backed by cache }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/cache2.h b/array/zfp/cache2.h index 5b63bd9f3..332c51707 100644 --- a/array/zfp/cache2.h +++ b/array/zfp/cache2.h @@ -4,6 +4,7 @@ #include "cache.h" namespace zfp { +namespace internal { template class BlockCache2 { @@ -40,7 +41,7 @@ class BlockCache2 { // flush cache by compressing all modified cached blocks void flush() const { - for (typename zfp::Cache::const_iterator p = cache.first(); p; p++) { + for (typename zfp::internal::Cache::const_iterator p = cache.first(); p; p++) { if (p->tag.dirty()) { size_t block_index = p->tag.index() - 1; store.encode(block_index, p->line->data()); @@ -165,7 +166,7 @@ class BlockCache2 { { CacheLine* p = 0; size_t block_index = store.block_index(i, j); - typename zfp::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); + typename zfp::internal::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); size_t stored_block_index = tag.index() - 1; if (stored_block_index != block_index) { // write back occupied cache line if it is dirty @@ -200,6 +201,7 @@ class BlockCache2 { Store& store; // store backed by cache }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/cache3.h b/array/zfp/cache3.h index 6e517bf7f..95258fd8d 100644 --- a/array/zfp/cache3.h +++ b/array/zfp/cache3.h @@ -4,6 +4,7 @@ #include "cache.h" namespace zfp { +namespace internal { template class BlockCache3 { @@ -40,7 +41,7 @@ class BlockCache3 { // flush cache by compressing all modified cached blocks void flush() const { - for (typename zfp::Cache::const_iterator p = cache.first(); p; p++) { + for (typename zfp::internal::Cache::const_iterator p = cache.first(); p; p++) { if (p->tag.dirty()) { size_t block_index = p->tag.index() - 1; store.encode(block_index, p->line->data()); @@ -171,7 +172,7 @@ class BlockCache3 { { CacheLine* p = 0; size_t block_index = store.block_index(i, j, k); - typename zfp::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); + typename zfp::internal::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); size_t stored_block_index = tag.index() - 1; if (stored_block_index != block_index) { // write back occupied cache line if it is dirty @@ -206,6 +207,7 @@ class BlockCache3 { Store& store; // store backed by cache }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/cache4.h b/array/zfp/cache4.h index 4c2c175c0..f16746858 100644 --- a/array/zfp/cache4.h +++ b/array/zfp/cache4.h @@ -4,6 +4,7 @@ #include "cache.h" namespace zfp { +namespace internal { template class BlockCache4 { @@ -40,7 +41,7 @@ class BlockCache4 { // flush cache by compressing all modified cached blocks void flush() const { - for (typename zfp::Cache::const_iterator p = cache.first(); p; p++) { + for (typename zfp::internal::Cache::const_iterator p = cache.first(); p; p++) { if (p->tag.dirty()) { size_t block_index = p->tag.index() - 1; store.encode(block_index, p->line->data()); @@ -177,7 +178,7 @@ class BlockCache4 { { CacheLine* p = 0; size_t block_index = store.block_index(i, j, k, l); - typename zfp::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); + typename zfp::internal::Cache::Tag tag = cache.access(p, (uint)block_index + 1, write); size_t stored_block_index = tag.index() - 1; if (stored_block_index != block_index) { // write back occupied cache line if it is dirty @@ -212,6 +213,7 @@ class BlockCache4 { Store& store; // store backed by cache }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/store.h b/array/zfp/store.h index 88e2f6363..554638051 100644 --- a/array/zfp/store.h +++ b/array/zfp/store.h @@ -6,6 +6,7 @@ #include "zfp/memory.h" namespace zfp { +namespace internal { // base class for block store template @@ -221,6 +222,7 @@ class BlockStore { Codec codec; // compression codec }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/store1.h b/array/zfp/store1.h index ed6311b20..0505c20c5 100644 --- a/array/zfp/store1.h +++ b/array/zfp/store1.h @@ -4,6 +4,7 @@ #include "zfp/store.h" namespace zfp { +namespace internal { // compressed block store for 1D array template @@ -133,6 +134,7 @@ class BlockStore1 : public BlockStore { size_t bx; // array dimensions in number of blocks }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/store2.h b/array/zfp/store2.h index 1a4a03e51..8dc60babc 100644 --- a/array/zfp/store2.h +++ b/array/zfp/store2.h @@ -4,6 +4,7 @@ #include "zfp/store.h" namespace zfp { +namespace internal { // compressed block store for 2D array template @@ -140,6 +141,7 @@ class BlockStore2 : public BlockStore { size_t bx, by; // array dimensions in number of blocks }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/store3.h b/array/zfp/store3.h index ac9ac2c73..7b26e196e 100644 --- a/array/zfp/store3.h +++ b/array/zfp/store3.h @@ -4,6 +4,7 @@ #include "zfp/store.h" namespace zfp { +namespace internal { // compressed block store for 3D array template @@ -147,6 +148,7 @@ class BlockStore3 : public BlockStore { size_t bx, by, bz; // array dimensions in number of blocks }; -} +} // internal +} // zfp #endif diff --git a/array/zfp/store4.h b/array/zfp/store4.h index 77c3f5c5f..75d404cfa 100644 --- a/array/zfp/store4.h +++ b/array/zfp/store4.h @@ -4,6 +4,7 @@ #include "zfp/store.h" namespace zfp { +namespace internal { // compressed block store for 4D array template @@ -154,6 +155,7 @@ class BlockStore4 : public BlockStore { size_t bx, by, bz, bw; // array dimensions in number of blocks }; -} +} // internal +} // zfp #endif diff --git a/array/zfparray1.h b/array/zfparray1.h index a17269e98..3b2b9e3df 100644 --- a/array/zfparray1.h +++ b/array/zfparray1.h @@ -30,8 +30,8 @@ class array1 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore1 store_type; - typedef BlockCache1 cache_type; + typedef zfp::internal::BlockStore1 store_type; + typedef zfp::internal::BlockCache1 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfparray2.h b/array/zfparray2.h index 11c3a2dfa..8c93a146a 100644 --- a/array/zfparray2.h +++ b/array/zfparray2.h @@ -30,8 +30,8 @@ class array2 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore2 store_type; - typedef BlockCache2 cache_type; + typedef zfp::internal::BlockStore2 store_type; + typedef zfp::internal::BlockCache2 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfparray3.h b/array/zfparray3.h index fd7657980..be47abe47 100644 --- a/array/zfparray3.h +++ b/array/zfparray3.h @@ -30,8 +30,8 @@ class array3 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore3 store_type; - typedef BlockCache3 cache_type; + typedef zfp::internal::BlockStore3 store_type; + typedef zfp::internal::BlockCache3 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfparray4.h b/array/zfparray4.h index e4aaeb9b5..9349d3afa 100644 --- a/array/zfparray4.h +++ b/array/zfparray4.h @@ -30,8 +30,8 @@ class array4 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore4 store_type; - typedef BlockCache4 cache_type; + typedef zfp::internal::BlockStore4 store_type; + typedef zfp::internal::BlockCache4 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfpcarray1.h b/array/zfpcarray1.h index e5a138de9..e77015f63 100644 --- a/array/zfpcarray1.h +++ b/array/zfpcarray1.h @@ -30,8 +30,8 @@ class const_array1 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore1 store_type; - typedef BlockCache1 cache_type; + typedef zfp::internal::BlockStore1 store_type; + typedef zfp::internal::BlockCache1 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfpcarray2.h b/array/zfpcarray2.h index 422f0e82f..7da5d66d6 100644 --- a/array/zfpcarray2.h +++ b/array/zfpcarray2.h @@ -30,8 +30,8 @@ class const_array2 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore2 store_type; - typedef BlockCache2 cache_type; + typedef zfp::internal::BlockStore2 store_type; + typedef zfp::internal::BlockCache2 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfpcarray3.h b/array/zfpcarray3.h index e233e9a58..bdfe21635 100644 --- a/array/zfpcarray3.h +++ b/array/zfpcarray3.h @@ -30,8 +30,8 @@ class const_array3 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore3 store_type; - typedef BlockCache3 cache_type; + typedef zfp::internal::BlockStore3 store_type; + typedef zfp::internal::BlockCache3 cache_type; typedef typename Codec::header header; // accessor classes diff --git a/array/zfpcarray4.h b/array/zfpcarray4.h index f305cae62..b556a3876 100644 --- a/array/zfpcarray4.h +++ b/array/zfpcarray4.h @@ -30,8 +30,8 @@ class const_array4 : public array { typedef Scalar value_type; typedef Codec codec_type; typedef Index index_type; - typedef BlockStore4 store_type; - typedef BlockCache4 cache_type; + typedef zfp::internal::BlockStore4 store_type; + typedef zfp::internal::BlockCache4 cache_type; typedef typename Codec::header header; // accessor classes From 84fb3d88070c4b2065fd409778ba1181407bb1a4 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 14 Dec 2021 16:07:14 -0800 Subject: [PATCH 019/277] Make array::set(0) zero-initialize array --- array/zfparray1.h | 15 ++++++++++++--- array/zfparray2.h | 19 ++++++++++++++----- array/zfparray3.h | 23 ++++++++++++++++------- array/zfparray4.h | 27 ++++++++++++++++++--------- docs/source/arrays.rst | 2 +- 5 files changed, 61 insertions(+), 25 deletions(-) diff --git a/array/zfparray1.h b/array/zfparray1.h index 3b2b9e3df..a388fc4cf 100644 --- a/array/zfparray1.h +++ b/array/zfparray1.h @@ -183,10 +183,19 @@ class array1 : public array { void set(const value_type* p) { const size_t bx = store.block_size_x(); - const ptrdiff_t sx = 1; size_t block_index = 0; - for (size_t i = 0; i < bx; i++, p += 4) - cache.put_block(block_index++, p, sx); + if (p) { + // compress data stored at p + const ptrdiff_t sx = 1; + for (size_t i = 0; i < bx; i++, p += 4) + cache.put_block(block_index++, p, sx); + } + else { + // zero-initialize array + const value_type block[4] = {}; + while (block_index < bx) + cache.put_block(block_index++, block, 1); + } } // accessors diff --git a/array/zfparray2.h b/array/zfparray2.h index 8c93a146a..7fc285cde 100644 --- a/array/zfparray2.h +++ b/array/zfparray2.h @@ -195,12 +195,21 @@ class array2 : public array { { const size_t bx = store.block_size_x(); const size_t by = store.block_size_y(); - const ptrdiff_t sx = 1; - const ptrdiff_t sy = static_cast(nx); size_t block_index = 0; - for (size_t j = 0; j < by; j++, p += 4 * sx * ptrdiff_t(nx - bx)) - for (size_t i = 0; i < bx; i++, p += 4) - cache.put_block(block_index++, p, sx, sy); + if (p) { + // compress data stored at p + const ptrdiff_t sx = 1; + const ptrdiff_t sy = static_cast(nx); + for (size_t j = 0; j < by; j++, p += 4 * sx * ptrdiff_t(nx - bx)) + for (size_t i = 0; i < bx; i++, p += 4) + cache.put_block(block_index++, p, sx, sy); + } + else { + // zero-initialize array + const value_type block[4 * 4] = {}; + while (block_index < bx * by) + cache.put_block(block_index++, block, 1, 4); + } } // (i, j) accessors diff --git a/array/zfparray3.h b/array/zfparray3.h index be47abe47..8d5a6a853 100644 --- a/array/zfparray3.h +++ b/array/zfparray3.h @@ -204,14 +204,23 @@ class array3 : public array { const size_t bx = store.block_size_x(); const size_t by = store.block_size_y(); const size_t bz = store.block_size_z(); - const ptrdiff_t sx = 1; - const ptrdiff_t sy = static_cast(nx); - const ptrdiff_t sz = static_cast(nx * ny); size_t block_index = 0; - for (size_t k = 0; k < bz; k++, p += 4 * sy * ptrdiff_t(ny - by)) - for (size_t j = 0; j < by; j++, p += 4 * sx * ptrdiff_t(nx - bx)) - for (size_t i = 0; i < bx; i++, p += 4) - cache.put_block(block_index++, p, sx, sy, sz); + if (p) { + // compress data stored at p + const ptrdiff_t sx = 1; + const ptrdiff_t sy = static_cast(nx); + const ptrdiff_t sz = static_cast(nx * ny); + for (size_t k = 0; k < bz; k++, p += 4 * sy * ptrdiff_t(ny - by)) + for (size_t j = 0; j < by; j++, p += 4 * sx * ptrdiff_t(nx - bx)) + for (size_t i = 0; i < bx; i++, p += 4) + cache.put_block(block_index++, p, sx, sy, sz); + } + else { + // zero-initialize array + const value_type block[4 * 4 * 4] = {}; + while (block_index < bx * by * bz) + cache.put_block(block_index++, block, 1, 4, 16); + } } // (i, j, k) accessors diff --git a/array/zfparray4.h b/array/zfparray4.h index 9349d3afa..dbebb9015 100644 --- a/array/zfparray4.h +++ b/array/zfparray4.h @@ -213,16 +213,25 @@ class array4 : public array { const size_t by = store.block_size_y(); const size_t bz = store.block_size_z(); const size_t bw = store.block_size_w(); - const ptrdiff_t sx = 1; - const ptrdiff_t sy = static_cast(nx); - const ptrdiff_t sz = static_cast(nx * ny); - const ptrdiff_t sw = static_cast(nx * ny * nz); size_t block_index = 0; - for (size_t l = 0; l < bw; l++, p += 4 * sz * ptrdiff_t(nz - bz)) - for (size_t k = 0; k < bz; k++, p += 4 * sy * ptrdiff_t(ny - by)) - for (size_t j = 0; j < by; j++, p += 4 * sx * ptrdiff_t(nx - bx)) - for (size_t i = 0; i < bx; i++, p += 4) - cache.put_block(block_index++, p, sx, sy, sz, sw); + if (p) { + // compress data stored at p + const ptrdiff_t sx = 1; + const ptrdiff_t sy = static_cast(nx); + const ptrdiff_t sz = static_cast(nx * ny); + const ptrdiff_t sw = static_cast(nx * ny * nz); + for (size_t l = 0; l < bw; l++, p += 4 * sz * ptrdiff_t(nz - bz)) + for (size_t k = 0; k < bz; k++, p += 4 * sy * ptrdiff_t(ny - by)) + for (size_t j = 0; j < by; j++, p += 4 * sx * ptrdiff_t(nx - bx)) + for (size_t i = 0; i < bx; i++, p += 4) + cache.put_block(block_index++, p, sx, sy, sz, sw); + } + else { + // zero-initialize array + const value_type block[4 * 4 * 4 * 4] = {}; + while (block_index < bx * by * bz * bw) + cache.put_block(block_index++, block, 1, 4, 16, 64); + } } // (i, j, k) accessors diff --git a/docs/source/arrays.rst b/docs/source/arrays.rst index 297674cb9..f0574534f 100644 --- a/docs/source/arrays.rst +++ b/docs/source/arrays.rst @@ -243,7 +243,7 @@ in the base class. Initialize array by copying and compressing data stored at *p*. The uncompressed data is assumed to be stored as in the :cpp:func:`get` - method. + method. If *p* = 0, then the array is zero-initialized. ---- From 31902956b119c1052eb4f8271678db803752bdc9 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 10 Nov 2021 16:30:34 -0800 Subject: [PATCH 020/277] remove unneeded test code --- tests/src/misc/testZfpHeader.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/src/misc/testZfpHeader.c b/tests/src/misc/testZfpHeader.c index 3ced7c45e..3986a739b 100644 --- a/tests/src/misc/testZfpHeader.c +++ b/tests/src/misc/testZfpHeader.c @@ -155,6 +155,7 @@ when_zfpFieldMetadataCalled_onInvalidSize_expect_ZFP_META_NULL(void **state) zfp_field* field = bundle->field; uint64 metadata = zfp_field_metadata(field); + // setup uses a 2d field field->nx = 1 << 25; field->ny = 1 << 25; @@ -168,10 +169,8 @@ when_zfpFieldSetMetadataCalled_forInvalidMeta_expect_false(void **state) { struct setupVars *bundle = *state; zfp_field* field = bundle->field; - uint64 metadata = zfp_field_metadata(field); uint64 meta = 1ULL << (ZFP_META_BITS + 1); - zfp_bool status = zfp_field_set_metadata(field, meta); assert_int_equal(status, zfp_false); From afac3f94f1c80ea1b1ea05b68a853342e6ef6839 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 10 Nov 2021 16:45:42 -0800 Subject: [PATCH 021/277] made test output more verbose --- .github/workflows/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c414debcc..45350a90f 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -77,7 +77,7 @@ jobs: - name: Run Tests id: test working-directory: ${{github.workspace}}/build - run: ctest -C ${{env.BUILD_TYPE}} + run: ctest -C ${{env.BUILD_TYPE}} -VV # Interactive Debug -> see: https://github.com/mxschmitt/action-tmate #- name: Setup Debug Session From d42fb0eea20392d8f3335422164b8f9a9547bf92 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 15 Dec 2021 11:24:28 -0800 Subject: [PATCH 022/277] enable actions pytests and split py adv decompress test into bytes/memview versions --- .github/workflows/main.yml | 2 +- tests/python/test_numpy.py | 40 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 45350a90f..5dde66183 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -68,7 +68,7 @@ jobs: - name: Run CMake id: cmake - run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=${{matrix.omp}} + run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=${{matrix.omp}} -DBUILD_ZFPY=ON - name: Build id: build diff --git a/tests/python/test_numpy.py b/tests/python/test_numpy.py index 1e52abcd5..e35ed043d 100644 --- a/tests/python/test_numpy.py +++ b/tests/python/test_numpy.py @@ -48,6 +48,46 @@ def test_different_dtypes(self): self.lossless_round_trip(array) def test_advanced_decompression_checksum(self): + ndims = 2 + ztype = zfpy.type_float + random_array = test_utils.getRandNumpyArray(ndims, ztype) + mode = zfpy.mode_fixed_accuracy + compress_param_num = 1 + compression_kwargs = { + "tolerance": test_utils.computeParameterValue( + mode, + compress_param_num + ), + } + compressed_array = zfpy.compress_numpy( + random_array, + write_header=False, + **compression_kwargs + ) + + # Decompression using the "advanced" interface which enforces no header, + # and the user must provide all the metadata + decompressed_array = np.empty_like(random_array) + zfpy._decompress( + compressed_array, + ztype, + random_array.shape, + out=decompressed_array, + **compression_kwargs + ) + decompressed_array_dims = decompressed_array.shape + tuple(0 for i in range(4 - decompressed_array.ndim)) + decompressed_checksum = test_utils.getChecksumDecompArray( + decompressed_array_dims, + ztype, + mode, + compress_param_num + ) + actual_checksum = test_utils.hashNumpyArray( + decompressed_array + ) + self.assertEqual(decompressed_checksum, actual_checksum) + + def test_memview_advanced_decompression_checksum(self): ndims = 2 ztype = zfpy.type_float random_array = test_utils.getRandNumpyArray(ndims, ztype) From 50774dcfdf8fe1069f212936b05b7e8454e57499 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Wed, 15 Dec 2021 12:01:53 -0800 Subject: [PATCH 023/277] Enable cython action temporarily disable win tests --- .github/workflows/main.yml | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 5dde66183..9437e52d0 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -20,11 +20,11 @@ jobs: omp: ON target: all - - os: windows-latest - cxx_compiler: msbuild - c_compiler: msbuild - omp: ON - target: ALL_BUILD + #- os: windows-latest + # cxx_compiler: msbuild + # c_compiler: msbuild + # omp: ON + # target: ALL_BUILD #- os: windows-latest # cxx_compiler: x86_64-w64-mingw32-g++ @@ -47,6 +47,17 @@ jobs: steps: - uses: actions/checkout@v2 + - name: Setup Python + uses: actions/setup-python@v2 + with: + python-version: '3.x' + architecture: x64 + + - name: Install zfpy dependencies + run: | + python -m pip install cython + python -m pip install numpy + - name: Setup MSBuild (Windows) id: msbuild if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} From 7ad84769498b66a3dccfbe19380d5bb2f8b71ccf Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Wed, 15 Dec 2021 12:07:20 -0800 Subject: [PATCH 024/277] Add python build hints to tests --- .github/workflows/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 9437e52d0..6751be240 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -79,7 +79,7 @@ jobs: - name: Run CMake id: cmake - run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=${{matrix.omp}} -DBUILD_ZFPY=ON + run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=${{matrix.omp}} -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") - name: Build id: build From 64e5db65e1dc93560a87eb8d2d95058ed851e634 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 15 Dec 2021 17:41:05 -0800 Subject: [PATCH 025/277] Update python cmake build scripts --- .github/workflows/main.yml | 12 ++ python/CMakeLists.txt | 4 +- python/eyescale-cmake/FindNumPy.cmake | 41 ------- python/eyescale-cmake/LICENSE.txt | 26 ----- python/scikit-build-cmake/FindCython.cmake | 9 +- python/scikit-build-cmake/FindNumPy.cmake | 106 ++++++++++++++++++ .../FindPythonExtensions.cmake | 20 ++-- python/scikit-build-cmake/LICENSE | 3 - python/scikit-build-cmake/UseCython.cmake | 32 +++--- ...targetLinkLibrariesWithDynamicLookup.cmake | 2 +- tests/python/CMakeLists.txt | 3 +- 11 files changed, 156 insertions(+), 102 deletions(-) delete mode 100644 python/eyescale-cmake/FindNumPy.cmake delete mode 100644 python/eyescale-cmake/LICENSE.txt create mode 100644 python/scikit-build-cmake/FindNumPy.cmake diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 6751be240..70b6b0412 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -6,6 +6,7 @@ jobs: build: runs-on: ${{matrix.os}} strategy: + fail-fast: false matrix: include: - os: ubuntu-latest @@ -46,6 +47,17 @@ jobs: target: all steps: - uses: actions/checkout@v2 + + - name: Setup Python + uses: actions/setup-python@v2 + with: + python-version: '3.x' + architecture: x64 + + - name: Install zfpy dependencies + run: | + python -m pip install cython + python -m pip install numpy - name: Setup Python uses: actions/setup-python@v2 diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index 76f931730..c6f6e27c0 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -1,5 +1,4 @@ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_LIST_DIR}/scikit-build-cmake) -set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_LIST_DIR}/eyescale-cmake) include(UseCython) include(FindPythonExtensions) include(FindNumPy) @@ -11,7 +10,7 @@ find_package(Cython REQUIRED) find_package(NumPy REQUIRED) include_directories(${ZFP_SOURCE_DIR}/include) -include_directories(${PYTHON_NUMPY_INCLUDE_DIR}) +include_directories(${NumPy_INCLUDE_DIR}) add_cython_target(zfpy zfpy.pyx C) add_library(zfpy MODULE ${zfpy}) @@ -21,6 +20,7 @@ python_extension_module(zfpy) # Build to the currrent binary dir to avoid conflicts with other libraries named zfp set(PYLIB_BUILD_DIR "${CMAKE_BINARY_DIR}/bin" CACHE PATH "Directory where zfp python library will be built") set_target_properties(zfpy PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${PYLIB_BUILD_DIR}) + # Install to the typical python module directory set(python_install_lib_dir "lib/python${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR}/site-packages/") install(TARGETS zfpy LIBRARY DESTINATION ${python_install_lib_dir}) diff --git a/python/eyescale-cmake/FindNumPy.cmake b/python/eyescale-cmake/FindNumPy.cmake deleted file mode 100644 index 8aba4e696..000000000 --- a/python/eyescale-cmake/FindNumPy.cmake +++ /dev/null @@ -1,41 +0,0 @@ -# Find the Python NumPy package -# PYTHON_NUMPY_INCLUDE_DIR -# PYTHON_NUMPY_FOUND -# will be set by this script - -# cmake_minimum_required(VERSION 2.6) - -if(NOT PYTHON_EXECUTABLE) - if(NumPy_FIND_QUIETLY) - find_package(PythonInterp QUIET) - else() - find_package(PythonInterp) - set(__numpy_out 1) - endif() -endif() - -if (PYTHON_EXECUTABLE) - # Find out the include path - execute_process( - COMMAND "${PYTHON_EXECUTABLE}" -c - "from __future__ import print_function\ntry: import numpy; print(numpy.get_include(), end='')\nexcept:pass\n" - OUTPUT_VARIABLE __numpy_path) - # And the version - execute_process( - COMMAND "${PYTHON_EXECUTABLE}" -c - "from __future__ import print_function\ntry: import numpy; print(numpy.__version__, end='')\nexcept:pass\n" - OUTPUT_VARIABLE __numpy_version) -elseif(__numpy_out) - message(STATUS "Python executable not found.") -endif(PYTHON_EXECUTABLE) - -find_path(PYTHON_NUMPY_INCLUDE_DIR numpy/arrayobject.h - HINTS "${__numpy_path}" "${PYTHON_INCLUDE_PATH}" NO_DEFAULT_PATH) - -if(PYTHON_NUMPY_INCLUDE_DIR) - set(PYTHON_NUMPY_FOUND 1 CACHE INTERNAL "Python numpy found") -endif(PYTHON_NUMPY_INCLUDE_DIR) - -include(FindPackageHandleStandardArgs) -find_package_handle_standard_args(NumPy REQUIRED_VARS PYTHON_NUMPY_INCLUDE_DIR - VERSION_VAR __numpy_version) diff --git a/python/eyescale-cmake/LICENSE.txt b/python/eyescale-cmake/LICENSE.txt deleted file mode 100644 index 307d54e59..000000000 --- a/python/eyescale-cmake/LICENSE.txt +++ /dev/null @@ -1,26 +0,0 @@ -Unless otherwise noted in the file, all files in this directory are -licensed under the BSD license, reproduced below. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - -- Redistributions of source code must retain the above copyright notice, this - list of conditions and the following disclaimer. -- Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. -- Neither the name of Eyescale Software GmbH nor the names of its - contributors may be used to endorse or promote products derived from this - software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE -LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR -CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF -SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS -INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN -CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) -ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE -POSSIBILITY OF SUCH DAMAGE. diff --git a/python/scikit-build-cmake/FindCython.cmake b/python/scikit-build-cmake/FindCython.cmake index 3d58c4f00..5f2ce6e09 100644 --- a/python/scikit-build-cmake/FindCython.cmake +++ b/python/scikit-build-cmake/FindCython.cmake @@ -13,7 +13,7 @@ # ``CYTHON_FOUND`` # true if the program was found # -# For more information on the Cython project, see http://cython.org/. +# For more information on the Cython project, see https://cython.org/. # # *Cython is a language that makes writing C extensions for the Python language # as easy as Python itself.* @@ -56,7 +56,8 @@ if(CYTHON_EXECUTABLE) OUTPUT_VARIABLE CYTHON_version_output ERROR_VARIABLE CYTHON_version_error RESULT_VARIABLE CYTHON_version_result - OUTPUT_STRIP_TRAILING_WHITESPACE) + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_STRIP_TRAILING_WHITESPACE) if(NOT ${CYTHON_version_result} EQUAL 0) set(_error_msg "Command \"${CYTHON_version_command}\" failed with") @@ -65,6 +66,10 @@ if(CYTHON_EXECUTABLE) else() if("${CYTHON_version_output}" MATCHES "^[Cc]ython version ([^,]+)") set(CYTHON_VERSION "${CMAKE_MATCH_1}") + else() + if("${CYTHON_version_error}" MATCHES "^[Cc]ython version ([^,]+)") + set(CYTHON_VERSION "${CMAKE_MATCH_1}") + endif() endif() endif() endif() diff --git a/python/scikit-build-cmake/FindNumPy.cmake b/python/scikit-build-cmake/FindNumPy.cmake new file mode 100644 index 000000000..cd78112b5 --- /dev/null +++ b/python/scikit-build-cmake/FindNumPy.cmake @@ -0,0 +1,106 @@ +#.rst: +# +# Find the include directory for ``numpy/arrayobject.h`` as well as other NumPy tools like ``conv-template`` and +# ``from-template``. +# +# This module sets the following variables: +# +# ``NumPy_FOUND`` +# True if NumPy was found. +# ``NumPy_INCLUDE_DIRS`` +# The include directories needed to use NumpPy. +# ``NumPy_VERSION`` +# The version of NumPy found. +# ``NumPy_CONV_TEMPLATE_EXECUTABLE`` +# Path to conv-template executable. +# ``NumPy_FROM_TEMPLATE_EXECUTABLE`` +# Path to from-template executable. +# +# The module will also explicitly define one cache variable: +# +# ``NumPy_INCLUDE_DIR`` +# +# .. note:: +# +# To support NumPy < v0.15.0 where ``from-template`` and ``conv-template`` are not declared as entry points, +# the module emulates the behavior of standalone executables by setting the corresponding variables with the +# path the the python interpreter and the path to the associated script. For example: +# :: +# +# set(NumPy_CONV_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/conv_template.py CACHE STRING "Command executing conv-template program" FORCE) +# +# set(NumPy_FROM_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/from_template.py CACHE STRING "Command executing from-template program" FORCE) +# + +if(NOT NumPy_FOUND) + set(_find_extra_args) + if(NumPy_FIND_REQUIRED) + list(APPEND _find_extra_args REQUIRED) + endif() + if(NumPy_FIND_QUIET) + list(APPEND _find_extra_args QUIET) + endif() + find_package(PythonInterp ${_find_extra_args}) + find_package(PythonLibs ${_find_extra_args}) + + find_program(NumPy_CONV_TEMPLATE_EXECUTABLE NAMES conv-template) + find_program(NumPy_FROM_TEMPLATE_EXECUTABLE NAMES from-template) + + if(PYTHON_EXECUTABLE) + execute_process(COMMAND "${PYTHON_EXECUTABLE}" + -c "import numpy; print(numpy.get_include())" + OUTPUT_VARIABLE _numpy_include_dir + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) + execute_process(COMMAND "${PYTHON_EXECUTABLE}" + -c "import numpy; print(numpy.__version__)" + OUTPUT_VARIABLE NumPy_VERSION + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) + + # XXX This is required to support NumPy < v0.15.0. See note in module documentation above. + if(NOT NumPy_CONV_TEMPLATE_EXECUTABLE) + execute_process(COMMAND "${PYTHON_EXECUTABLE}" + -c "from numpy.distutils import conv_template; print(conv_template.__file__)" + OUTPUT_VARIABLE _numpy_conv_template_file + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) + set(NumPy_CONV_TEMPLATE_EXECUTABLE "${PYTHON_EXECUTABLE}" "${_numpy_conv_template_file}" CACHE STRING "Command executing conv-template program" FORCE) + endif() + + # XXX This is required to support NumPy < v0.15.0. See note in module documentation above. + if(NOT NumPy_FROM_TEMPLATE_EXECUTABLE) + execute_process(COMMAND "${PYTHON_EXECUTABLE}" + -c "from numpy.distutils import from_template; print(from_template.__file__)" + OUTPUT_VARIABLE _numpy_from_template_file + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) + set(NumPy_FROM_TEMPLATE_EXECUTABLE "${PYTHON_EXECUTABLE}" "${_numpy_from_template_file}" CACHE STRING "Command executing from-template program" FORCE) + endif() + endif() +endif() + +find_path(NumPy_INCLUDE_DIR + numpy/arrayobject.h + PATHS "${_numpy_include_dir}" "${PYTHON_INCLUDE_DIR}" + PATH_SUFFIXES numpy/core/include + ) + +set(NumPy_INCLUDE_DIRS ${NumPy_INCLUDE_DIR}) + +# handle the QUIETLY and REQUIRED arguments and set NumPy_FOUND to TRUE if +# all listed variables are TRUE +include(FindPackageHandleStandardArgs) +find_package_handle_standard_args(NumPy + REQUIRED_VARS + NumPy_INCLUDE_DIR + NumPy_CONV_TEMPLATE_EXECUTABLE + NumPy_FROM_TEMPLATE_EXECUTABLE + VERSION_VAR NumPy_VERSION + ) + +mark_as_advanced(NumPy_INCLUDE_DIR) diff --git a/python/scikit-build-cmake/FindPythonExtensions.cmake b/python/scikit-build-cmake/FindPythonExtensions.cmake index 9a3d76a0c..78d82f5f4 100644 --- a/python/scikit-build-cmake/FindPythonExtensions.cmake +++ b/python/scikit-build-cmake/FindPythonExtensions.cmake @@ -104,9 +104,10 @@ # [HEADER_OUTPUT_VAR ] # [INCLUDE_DIR_OUTPUT_VAR ]) # +# without the extension is used as the logical name. If only ```` is +# # If only ```` is provided, and it ends in the ".h" extension, then it # is assumed to be the ````. The filename of the header file -# without the extension is used as the logical name. If only ```` is # provided, and it does not end in the ".h" extension, then the # ```` is assumed to ``.h``. # @@ -200,7 +201,7 @@ # FORWARD_DECL_MODULES_VAR fdecl_module_list) # # # module2 -- dynamically linked -# include_directories({Boost_INCLUDE_DIRS}) +# include_directories(${Boost_INCLUDE_DIRS}) # add_library(module2 SHARED boost_module2.cxx) # target_link_libraries(module2 ${Boost_LIBRARIES}) # python_extension_module(module2 @@ -209,7 +210,7 @@ # # # module3 -- loaded at runtime # add_cython_target(module3a.pyx) -# add_library(module1 MODULE ${module3a} module3b.cxx) +# add_library(module3 MODULE ${module3a} module3b.cxx) # target_link_libraries(module3 ${Boost_LIBRARIES}) # python_extension_module(module3 # LINKED_MODULES_VAR linked_module_list @@ -254,7 +255,6 @@ import os import os.path import site import sys -import sysconfig result = None rel_result = None @@ -282,13 +282,17 @@ for candidate in candidates: rel_result = rel_candidate break +ext_suffix_var = 'SO' +if sys.version_info[:2] >= (3, 5): + ext_suffix_var = 'EXT_SUFFIX' + sys.stdout.write(\";\".join(( os.sep, os.pathsep, sys.prefix, result, rel_result, - sysconfig.get_config_var('SO') + distutils.sysconfig.get_config_var(ext_suffix_var) ))) ") @@ -332,7 +336,7 @@ function(_set_python_extension_symbol_visibility _target) set_target_properties(${_target} PROPERTIES LINK_FLAGS "/EXPORT:${_modinit_prefix}${_target}" ) - elseif("${CMAKE_C_COMPILER_ID}" STREQUAL "GNU") + elseif("${CMAKE_C_COMPILER_ID}" STREQUAL "GNU" AND NOT ${CMAKE_SYSTEM_NAME} MATCHES "Darwin") set(_script_path ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/${_target}-version-script.map ) @@ -340,7 +344,7 @@ function(_set_python_extension_symbol_visibility _target) "{global: ${_modinit_prefix}${_target}; local: *; };" ) set_property(TARGET ${_target} APPEND_STRING PROPERTY LINK_FLAGS - " -Wl,--version-script=${_script_path}" + " -Wl,--version-script=\"${_script_path}\"" ) endif() endfunction() @@ -423,7 +427,7 @@ function(python_extension_module _target) target_link_libraries_with_dynamic_lookup(${_target} ${PYTHON_LIBRARIES}) if(_is_module_lib) - #_set_python_extension_symbol_visibility(${_altname}) + _set_python_extension_symbol_visibility(${_target}) endif() endif() endfunction() diff --git a/python/scikit-build-cmake/LICENSE b/python/scikit-build-cmake/LICENSE index 73a9db0f2..3a85dcffc 100644 --- a/python/scikit-build-cmake/LICENSE +++ b/python/scikit-build-cmake/LICENSE @@ -1,6 +1,3 @@ -Unless otherwise noted in the file, all files in this directory are -licensed under the MIT license, reproduced below. - The MIT License (MIT) Copyright (c) 2014 Mike Sarahan diff --git a/python/scikit-build-cmake/UseCython.cmake b/python/scikit-build-cmake/UseCython.cmake index 9a5966488..2c40bd7b5 100644 --- a/python/scikit-build-cmake/UseCython.cmake +++ b/python/scikit-build-cmake/UseCython.cmake @@ -43,7 +43,7 @@ # ``PY2 | PY3`` # Force compilation using either Python-2 or Python-3 syntax and code # semantics. By default, Python-2 syntax and semantics are used if the major -# version of Python found is 2. Otherwise, Python-3 syntax and sematics are +# version of Python found is 2. Otherwise, Python-3 syntax and semantics are # used. # # ``OUTPUT_VAR `` @@ -56,13 +56,13 @@ # ```` # The path of the generated source file. # -# Cache variables that effect the behavior include: +# Cache variables that affect the behavior include: # # ``CYTHON_ANNOTATE`` -# whether to create an annotated .html file when compiling +# Whether to create an annotated .html file when compiling. # # ``CYTHON_FLAGS`` -# additional flags to pass to the Cython compiler +# Additional flags to pass to the Cython compiler. # # Example usage # ^^^^^^^^^^^^^ @@ -101,7 +101,6 @@ set(CYTHON_ANNOTATE OFF set(CYTHON_FLAGS "" CACHE STRING "Extra flags to the cython compiler.") mark_as_advanced(CYTHON_ANNOTATE CYTHON_FLAGS) -string(REGEX REPLACE " " ";" CYTHON_FLAGS_LIST "${CYTHON_FLAGS}") find_package(PythonLibs REQUIRED) @@ -138,6 +137,14 @@ function(add_cython_target _name) set(_embed_main FALSE) + if("C" IN_LIST languages) + set(_output_syntax "C") + elseif("CXX" IN_LIST languages) + set(_output_syntax "CXX") + else() + message(FATAL_ERROR "Either C or CXX must be enabled to use Cython") + endif() + if("${PYTHONLIBS_VERSION_STRING}" MATCHES "^2.") set(_input_syntax "PY2") else() @@ -323,21 +330,11 @@ function(add_cython_target _name) set(annotate_arg "--annotate") endif() - set(no_docstrings_arg "") - set(embed_signature_arg "") - if(CMAKE_BUILD_TYPE STREQUAL "MinSizeRel") - set(no_docstrings_arg "--no-docstrings") - else() - set(embed_signature_arg "-Xembedsignature=True") - endif() - set(cython_debug_arg "") - set(embed_pos_arg "") set(line_directives_arg "") if(CMAKE_BUILD_TYPE STREQUAL "Debug" OR CMAKE_BUILD_TYPE STREQUAL "RelWithDebInfo") set(cython_debug_arg "--gdb") - set(embed_pos_arg "--embed-positions") set(line_directives_arg "--line-directives") endif() @@ -352,12 +349,13 @@ function(add_cython_target _name) list(REMOVE_DUPLICATES pxd_dependencies) list(REMOVE_DUPLICATES c_header_dependencies) + string(REGEX REPLACE " " ";" CYTHON_FLAGS_LIST "${CYTHON_FLAGS}") + # Add the command to run the compiler. add_custom_command(OUTPUT ${generated_file} COMMAND ${CYTHON_EXECUTABLE} ARGS ${cxx_arg} ${include_directory_arg} ${py_version_arg} - ${embed_arg} ${annotate_arg} ${no_docstrings_arg} - ${cython_debug_arg} ${embed_pos_arg} ${embed_signature_arg} + ${embed_arg} ${annotate_arg} ${cython_debug_arg} ${line_directives_arg} ${CYTHON_FLAGS_LIST} ${pyx_location} --output-file ${generated_file} DEPENDS ${_source_file} diff --git a/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake b/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake index 020fc404a..6199ed5ea 100644 --- a/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake +++ b/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake @@ -362,7 +362,7 @@ function(_test_weak_link_project file(APPEND "${test_project_src_dir}/main.c" " goto done; error: - fprintf(stderr, \"Error occured:\\n %s\\n\", dlerror()); + fprintf(stderr, \"Error occurred:\\n %s\\n\", dlerror()); result = 1; done: diff --git a/tests/python/CMakeLists.txt b/tests/python/CMakeLists.txt index 89fce2d1b..a6b51eca0 100644 --- a/tests/python/CMakeLists.txt +++ b/tests/python/CMakeLists.txt @@ -1,5 +1,4 @@ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_SOURCE_DIR}/python/scikit-build-cmake) -set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_SOURCE_DIR}/python/eyescale-cmake) find_package(PythonInterp REQUIRED) find_package(PythonLibs REQUIRED) @@ -9,7 +8,7 @@ find_package(NumPy REQUIRED) include_directories(${ZFP_SOURCE_DIR}/include) include_directories(${ZFP_SOURCE_DIR}/python) -include_directories(${PYTHON_NUMPY_INCLUDE_DIR}) +include_directories(${NumPy_INCLUDE_DIR}) include_directories(${ZFP_SOURCE_DIR}/tests/python) include_directories(${ZFP_SOURCE_DIR}/tests/utils) From fd573184642e46a7300421679aeac20b496622ed Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Tue, 18 Jan 2022 16:30:01 -0800 Subject: [PATCH 026/277] Move status badge from travis to actions --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 34ae2de04..3ca6903ff 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ ZFP === -[![Travis CI Build Status](https://travis-ci.com/LLNL/zfp.svg?branch=develop)](https://travis-ci.com/LLNL/zfp) +[![Github Actions Build Status](https://github.com/LLNL/zfp/actions/workflows/main.yml/badge.svg?branch=develop)](https://github.com/LLNL/zfp/actions/workflows/main.yml) [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) [![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release0.5.5)](https://zfp.readthedocs.io/en/release0.5.5/?badge=release0.5.5) [![Code Coverage](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg)](https://codecov.io/gh/LLNL/zfp) From a674bf58bb2b6f0926faa12409a01e08efe11332 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 19 Jan 2022 11:52:38 -0800 Subject: [PATCH 027/277] Remove Travis CI files --- .travis.yml | 280 --------------------------------------------- cmake/travis.cmake | 87 -------------- travis.sh | 60 ---------- 3 files changed, 427 deletions(-) delete mode 100644 .travis.yml delete mode 100644 cmake/travis.cmake delete mode 100755 travis.sh diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 19bbab21d..000000000 --- a/.travis.yml +++ /dev/null @@ -1,280 +0,0 @@ -language: - - generic - -matrix: - include: - - os: osx - osx_image: xcode8.3 - env: - - MATRIX_EVAL="CC=clang && CXX=clang++ && PYTHON_VERSION=2.7" - - - os: osx - osx_image: xcode8.3 - env: - - MATRIX_EVAL="CC=clang && CXX=clang++ && PYTHON_VERSION=3.5" - - - os: osx - osx_image: xcode8.3 - env: - - MATRIX_EVAL="CC=gcc-4.9 && CXX=g++-4.9 && PYTHON_VERSION=2.7" - - - os: osx - osx_image: xcode8.3 - env: - - MATRIX_EVAL="CC=gcc-4.9 && CXX=g++-4.9 && PYTHON_VERSION=3.5" - - #- os: linux - # dist: xenial - # compiler: gcc-6 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-6 - # - g++-6 - # - gfortran-6 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-6' CXX='g++-6' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' COVERAGE='ON' - - #- os: linux - # dist: xenial - # compiler: clang-3.6 - # addons: &clang36 - # apt: - # sources: ['ubuntu-toolchain-r-test', 'llvm-toolchain-trusty'] - # packages: - # - clang-3.6 - # - g++-7 - # - gfortran-6 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='clang-3.6' CXX='clang++-3.6' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: clang-4.0 - # before_install: - # - export LD_LIBRARY_PATH=/usr/local/clang/lib:$LD_LIBRARY_PATH - # addons: &clang40 - # apt: - # sources: ['ubuntu-toolchain-r-test', 'llvm-toolchain-trusty-4.0'] - # packages: - # - clang-4.0 - # - g++-7 - # - gfortran-6 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='clang-4.0' CXX='clang++-4.0' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: gcc-4.4 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-4.4 - # - g++-4.4 - # - gfortran-4.4 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-4.4' CXX='g++-4.4' FC='gfortran-4.4' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: gcc-4.7 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-4.7 - # - g++-4.7 - # - gfortran-4.7 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-4.7' CXX='g++-4.7' FC='gfortran-4.7' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: gcc-4.8 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-4.8 - # - g++-4.8 - # - gfortran-4.8 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-4.8' CXX='g++-4.8' FC='gfortran-4.8' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: gcc-4.9 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-4.9 - # - g++-4.9 - # - gfortran-4.9 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-4.9' CXX='g++-4.9' FC='gfortran-4.9' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: gcc-6 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-6 - # - g++-6 - # - gfortran-6 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-6' CXX='g++-6' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' C_STANDARD='90' - - #- os: linux - # dist: xenial - # compiler: gcc-6 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-6 - # - g++-6 - # - gfortran-6 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-6' CXX='g++-6' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' C_STANDARD='11' - - #- os: linux - # dist: xenial - # compiler: gcc-6 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-6 - # - g++-6 - # - gfortran-6 - # - libpython3.5-dev - # - python3-numpy - # - python3-pip - # env: CC='gcc-6' CXX='g++-6' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' CXX_STANDARD='11' - - #- os: linux - # dist: xenial - # compiler: gcc-6 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-6 - # - g++-6 - # - gfortran-6 - # - libpython3.5 - # - python3-numpy - # - python3-pip - # env: CC='gcc-6' CXX='g++-6' FC='gfortran-6' FORTRAN_STANDARD='2003' PYTHON_VERSION='3.5' CXX_STANDARD='14' - - #- os: linux - # dist: xenial - # compiler: gcc-6 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-6 - # - g++-6 - # - gfortran-6 - # - libpython3.5 - # - python3-numpy - # - python3-pip - # env: CC='gcc-6' CXX='g++-6' FC='gfortran-6' FORTRAN_STANDARD='2008' PYTHON_VERSION='3.5' - - #- os: linux - # dist: xenial - # compiler: gcc-7 - # addons: - # apt: - # sources: ubuntu-toolchain-r-test - # packages: - # - gcc-7 - # - g++-7 - # - gfortran-7 - # - libpython3.5 - # - python3-numpy - # - python3-pip - # env: CC='gcc-7' CXX='g++-7' FC='gfortran-7' FORTRAN_STANDARD='2008' PYTHON_VERSION='3.5' - -before_install: - - eval "${MATRIX_EVAL}" - -script: - - if [ "$TRAVIS_OS_NAME" == "osx" ]; then pyenv root; fi - - | - if [ "$TRAVIS_OS_NAME" = "osx" ] && [ "$PYTHON_VERSION" = "2.7" ]; then - pyenv install 2.7.12; - export PYTHON_INCLUDE_DIR=$(pyenv root)/versions/2.7.12/include/python2.7; - export PYTHON_LIBRARY=$(pyenv root)/versions/2.7.12/lib/libpython2.7.dylib; - export PYTHON_EXECUTABLE=$(pyenv root)/versions/2.7.12/bin/python2.7; - fi - - | - if [ "$TRAVIS_OS_NAME" = "osx" ] && [ "$PYTHON_VERSION" = "3.5" ]; then - pyenv install 3.5.0; - export PYTHON_INCLUDE_DIR=$(pyenv root)/versions/3.5.0/include/python3.5m; - export PYTHON_LIBRARY=$(pyenv root)/versions/3.5.0/lib/libpython3.5m.a; - export PYTHON_EXECUTABLE=$(pyenv root)/versions/3.5.0/bin/python3.5m; - fi - - | - if [ "$TRAVIS_OS_NAME" == "osx" ]; then - $PYTHON_EXECUTABLE -m pip install --upgrade 'pip<21'; - $PYTHON_EXECUTABLE -m pip install -r ${TRAVIS_BUILD_DIR}/python/requirements.txt; - fi - - - | - if [ "$TRAVIS_OS_NAME" = "linux" ]; then - export PYTHON_EXECUTABLE=/usr/bin/python$PYTHON_VERSION; - source /etc/lsb-release; - fi - - | - if [ "$TRAVIS_OS_NAME" = "linux" ] && [ "$PYTHON_VERSION" = "2.7" ]; then - export PYTHON_INCLUDE_DIR=/usr/include/python2.7; - export PYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython2.7.so; - fi - - | - if [ "$TRAVIS_OS_NAME" = "linux" ] && [ "$PYTHON_VERSION" = "3.5" ]; then - export PYTHON_INCLUDE_DIR=/usr/include/python3.5m; - export PYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so; - sudo $PYTHON_EXECUTABLE -m pip install --upgrade 'pip<21'; - sudo $PYTHON_EXECUTABLE -m pip install --upgrade cython; - fi - - | - if [ "$TRAVIS_OS_NAME" = "linux" ] && [ "$DISTRIB_CODENAME" = "trusty" ] && [ "$PYTHON_VERSION" = "2.7" ]; then - sudo $PYTHON_EXECUTABLE -m pip install --upgrade 'pip<21'; - sudo $PYTHON_EXECUTABLE -m pip install -r ${TRAVIS_BUILD_DIR}/python/requirements.txt; - fi - - | - if [ "$TRAVIS_OS_NAME" = "linux" ] && [ "$DISTRIB_CODENAME" = "trusty" ] && [ "$PYTHON_VERSION" = "3.5" ]; then - echo "Python 3.5 not supported on Ubuntu Trusty"; - exit 1; - fi - - - printenv | grep PYTHON - - ./travis.sh - -after_success: - - if [[ -n "${COVERAGE}" ]]; then bash <(curl -s https://codecov.io/bash); fi diff --git a/cmake/travis.cmake b/cmake/travis.cmake deleted file mode 100644 index f2bf844b7..000000000 --- a/cmake/travis.cmake +++ /dev/null @@ -1,87 +0,0 @@ - -set(CTEST_SOURCE_DIRECTORY "$ENV{TRAVIS_BUILD_DIR}") -set(CTEST_BINARY_DIRECTORY "$ENV{TRAVIS_BUILD_DIR}/build") - -set(CTEST_COMMAND ctest) -include(${CTEST_SOURCE_DIRECTORY}/CTestConfig.cmake) -set(CTEST_SITE "travis") -set(CTEST_CMAKE_GENERATOR "Unix Makefiles") -set(CTEST_BUILD_NAME "$ENV{TRAVIS_BRANCH}-#$ENV{TRAVIS_JOB_NUMBER}") -set(cfg_options - -DCMAKE_C_STANDARD=${C_STANDARD} - -DCMAKE_CXX_STANDARD=${CXX_STANDARD} - -DBUILD_CFP=${BUILD_CFP} - -DBUILD_ZFPY=${BUILD_ZFPY} - -DBUILD_ZFORP=${BUILD_ZFORP} - -DZFP_WITH_OPENMP=${BUILD_OPENMP} - -DZFP_WITH_CUDA=${BUILD_CUDA} - ) - -# Add the variants to the testers name so that we can report multiple -# times from the same CI builder -if(BUILD_OPENMP) - set(CTEST_SITE "${CTEST_SITE}_openmp") -endif() - -if(BUILD_CUDA) - set(CTEST_SITE "${CTEST_SITE}_cuda") -endif() - -if(BUILD_CFP) - set(CTEST_SITE "${CTEST_SITE}_cfp") - - if(CFP_NAMESPACE) - list(APPEND cfg_options - -DCFP_NAMESPACE=${CFP_NAMESPACE} - ) - set(CTEST_SITE "${CTEST_SITE}namespace") - endif() -endif() - -if(BUILD_ZFPY) - set(CTEST_SITE "${CTEST_SITE}_zfpy$ENV{PYTHON_VERSION}") - list(APPEND cfg_options - -DPYTHON_INCLUDE_DIR=$ENV{PYTHON_INCLUDE_DIR} - -DPYTHON_LIBRARY=$ENV{PYTHON_LIBRARY} - -DPYTHON_EXECUTABLE=$ENV{PYTHON_EXECUTABLE} - ) -endif() - -if(BUILD_ZFORP) - set(CTEST_SITE "${CTEST_SITE}_zforp$ENV{FORTRAN_STANDARD}") - list(APPEND cfg_options - -DCMAKE_FORTRAN_FLAGS='-std=f$ENV{FORTRAN_STANDARD}' - ) -endif() - -if(WITH_COVERAGE) - list(APPEND cfg_options - -DCMAKE_C_FLAGS=-coverage - -DCMAKE_CXX_FLAGS=-coverage - -DCMAKE_Fortran_FLAGS=-coverage - ) - set(CTEST_SITE "${CTEST_SITE}_coverage") -endif() - -if(OMP_TESTS_ONLY) - list(APPEND cfg_options - -DZFP_OMP_TESTS_ONLY=1 - ) -endif() - -ctest_start(Experimental TRACK Travis) -ctest_configure(OPTIONS "${cfg_options}") -ctest_submit(PARTS Update Notes Configure) -ctest_build(FLAGS -j1) -ctest_submit(PARTS Build) -ctest_test(PARALLEL_LEVEL 6 RETURN_VALUE rv) -ctest_submit(PARTS Test) - -if(WITH_COVERAGE) - ctest_coverage() - ctest_submit(PARTS Coverage) -endif() - -if(NOT rv EQUAL 0) - message(FATAL_ERROR "Test failures occurred.") -endif() diff --git a/travis.sh b/travis.sh deleted file mode 100755 index e73383cde..000000000 --- a/travis.sh +++ /dev/null @@ -1,60 +0,0 @@ -#!/usr/bin/env sh -set -e - -# pass additional args in $1 (starting with whitespace character) -run_all () { - run_all_cmd="ctest -V -C Debug -DC_STANDARD=${C_STANDARD:-99} -DCXX_STANDARD=${CXX_STANDARD:-98} -S \"$TRAVIS_BUILD_DIR/cmake/travis.cmake\"" - eval "${run_all_cmd}$1" -} - -mkdir build -cd build - -# technically, flags are passed on to cmake/* and actually set there -BUILD_FLAGS="" - -if [ -n "${COVERAGE}" ]; then - # build (linux) - - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_UTILITIES=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_EXAMPLES=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_CFP=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_ZFPY=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_ZFORP=ON" - BUILD_FLAGS="$BUILD_FLAGS -DZFP_WITH_ALIGNED_ALLOC=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_OPENMP=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_CUDA=OFF" - BUILD_FLAGS="$BUILD_FLAGS -DWITH_COVERAGE=ON" - - run_all "$BUILD_FLAGS" -else - # build/test without OpenMP, with CFP (and custom namespace), with zfPy, with Fortran (linux only) - if [[ "$OSTYPE" == "darwin"* ]]; then - BUILD_ZFORP=OFF - else - BUILD_ZFORP=ON - fi - - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_UTILITIES=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_EXAMPLES=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_CFP=ON" - BUILD_FLAGS="$BUILD_FLAGS -DCFP_NAMESPACE=cfp2" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_ZFPY=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_ZFORP=$BUILD_ZFORP" - BUILD_FLAGS="$BUILD_FLAGS -DZFP_WITH_ALIGNED_ALLOC=ON" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_OPENMP=OFF" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_CUDA=OFF" - run_all "$BUILD_FLAGS" - - rm -rf ./* ; - - # if OpenMP available, start a 2nd build with it - if cmake ../tests/ci-utils/ ; then - rm -rf ./* ; - - # build/test with OpenMP - BUILD_FLAGS="" - BUILD_FLAGS="$BUILD_FLAGS -DBUILD_OPENMP=ON" - run_all "$BUILD_FLAGS" - fi -fi From ba2df4bcd380e87ec36d2f1527d3cef19906dfdd Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 20 Jan 2022 10:54:48 -0800 Subject: [PATCH 028/277] move gitlab cpu tests to new system --- tests/gitlab/gitlab-ci.yml | 2 - tests/gitlab/pascal-jobs.yml | 73 ++++++++++++++++++++++++++++++ tests/gitlab/pascal-templates.yml | 8 ++++ tests/gitlab/surface-jobs.yml | 72 ----------------------------- tests/gitlab/surface-templates.yml | 20 -------- 5 files changed, 81 insertions(+), 94 deletions(-) delete mode 100644 tests/gitlab/surface-jobs.yml delete mode 100644 tests/gitlab/surface-templates.yml diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 7c48b8ead..7594eb402 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -116,8 +116,6 @@ stages: ############ include: - - local: tests/gitlab/surface-templates.yml - - local: tests/gitlab/surface-jobs.yml - local: tests/gitlab/pascal-templates.yml - local: tests/gitlab/pascal-jobs.yml # - local: tests/gitlab/corona-templates.yml diff --git a/tests/gitlab/pascal-jobs.yml b/tests/gitlab/pascal-jobs.yml index 899dba02b..bfb968808 100644 --- a/tests/gitlab/pascal-jobs.yml +++ b/tests/gitlab/pascal-jobs.yml @@ -13,3 +13,76 @@ cuda-10.1.168_test: ci_test_regex: "Cuda" extends: [.pascal_test_gpu] needs: [cuda-10.1.168_build] + +########### +# CXX CPU # +########### + +cpp_gnu-7.3.0_build: + variables: + ci_cxx_cmp: "g++" + ci_c_cmp: "gcc" + ci_cmp_mod: "gcc/7.3.0" + extends: [.cpp, .pascal_build_cpu] + needs: [] + +cpp_gnu-7.3.0_test: + extends: [.pascal_test_cpu] + needs: [cpp_gnu-7.3.0_build] + + +cpp_clang-10.0.0_build: + variables: + ci_cxx_cmp: "clang++" + ci_c_cmp: "clang" + ci_cmp_mod: "clang/10.0.0" + extends: [.cpp, .pascal_build_cpu] + needs: [] + +cpp_clang-10.0.0_test: + extends: [.pascal_test_cpu] + needs: [cpp_clang-10.0.0_build] + + +cpp_intel-19.0.4_build: + variables: + ci_cxx_cmp: "icpc" + ci_c_cmp: "icc" + ci_cmp_mod: "intel/19.0.4" + extends: [.cpp, .pascal_build_cpu] + needs: [] + +cpp_intel-19.0.4_test: + extends: [.pascal_test_cpu] + needs: [cpp_intel-19.0.4_build] + + +cpp_pgi-21.1_build: + variables: + ci_cxx_cmp: "pgc++" + ci_c_cmp: "pgcc" + ci_cmp_mod: "pgi/21.1" + extends: [.cpp, .pascal_build_cpu] + needs: [] + +cpp_pgi-21.1_test: + extends: [.pascal_test_cpu] + needs: [cpp_pgi-21.1_build] + + +######### +# C CPU # +######### + +c_gnu-7.3.0_build: + variables: + ci_c_cmp: "gcc" + ci_cmp_mod: "gcc/7.3.0" + extends: [.c, .pascal_build_cpu] + needs: [] + +c_gnu-7.3.0_test: + variables: + ci_test_regex: "Cfp" + extends: [.pascal_test_cpu] + needs: [c_gnu-7.3.0_build] diff --git a/tests/gitlab/pascal-templates.yml b/tests/gitlab/pascal-templates.yml index 30b262696..1537088ea 100644 --- a/tests/gitlab/pascal-templates.yml +++ b/tests/gitlab/pascal-templates.yml @@ -10,3 +10,11 @@ variables: ci_test_regex: "." extends: [.test_gpu, .pascal_job] + +.pascal_build_cpu: + extends: [.build_cpu, .pascal_job] + +.pascal_test_cpu: + variables: + ci_test_regex: "." + extends: [.test_cpu, .pascal_job] diff --git a/tests/gitlab/surface-jobs.yml b/tests/gitlab/surface-jobs.yml deleted file mode 100644 index 5325c135a..000000000 --- a/tests/gitlab/surface-jobs.yml +++ /dev/null @@ -1,72 +0,0 @@ -########### -# CXX CPU # -########### - -cpp_gnu-7.3.0_build: - variables: - ci_cxx_cmp: "g++" - ci_c_cmp: "gcc" - ci_cmp_mod: "gcc/7.3.0" - extends: [.cpp, .surface_build_cpu] - needs: [] - -cpp_gnu-7.3.0_test: - extends: [.surface_test_cpu] - needs: [cpp_gnu-7.3.0_build] - - -cpp_clang-10.0.0_build: - variables: - ci_cxx_cmp: "clang++" - ci_c_cmp: "clang" - ci_cmp_mod: "clang/10.0.0" - extends: [.cpp, .surface_build_cpu] - needs: [] - -cpp_clang-10.0.0_test: - extends: [.surface_test_cpu] - needs: [cpp_clang-10.0.0_build] - - -cpp_intel-19.0.4_build: - variables: - ci_cxx_cmp: "icpc" - ci_c_cmp: "icc" - ci_cmp_mod: "intel/19.0.4" - extends: [.cpp, .surface_build_cpu] - needs: [] - -cpp_intel-19.0.4_test: - extends: [.surface_test_cpu] - needs: [cpp_intel-19.0.4_build] - - -cpp_pgi-21.1_build: - variables: - ci_cxx_cmp: "pgc++" - ci_c_cmp: "pgcc" - ci_cmp_mod: "pgi/21.1" - extends: [.cpp, .surface_build_cpu] - needs: [] - -cpp_pgi-21.1_test: - extends: [.surface_test_cpu] - needs: [cpp_pgi-21.1_build] - - -######### -# C CPU # -######### - -c_gnu-7.3.0_build: - variables: - ci_c_cmp: "gcc" - ci_cmp_mod: "gcc/7.3.0" - extends: [.c, .surface_build_cpu] - needs: [] - -c_gnu-7.3.0_test: - variables: - ci_test_regex: "Cfp" - extends: [.surface_test_cpu] - needs: [c_gnu-7.3.0_build] diff --git a/tests/gitlab/surface-templates.yml b/tests/gitlab/surface-templates.yml deleted file mode 100644 index ee838f3fc..000000000 --- a/tests/gitlab/surface-templates.yml +++ /dev/null @@ -1,20 +0,0 @@ -.surface_job: - tags: - - batch - - surface - -.surface_build_cpu: - extends: [.build_cpu, .surface_job] - -.surface_test_cpu: - variables: - ci_test_regex: "." - extends: [.test_cpu, .surface_job] - -.surface_build_gpu: - extends: [.build_gpu, .surface_job] - -.surface_test_gpu: - variables: - ci_test_regex: "." - extends: [.test_gpu, .surface_job] From 5fe5ed198888609df5beb513e36bfae967304832 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Thu, 20 Jan 2022 11:22:33 -0800 Subject: [PATCH 029/277] Change name of actions workflow --- .github/workflows/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 70b6b0412..eeb42d06c 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -1,4 +1,4 @@ -name: run tests +name: build on: push env: BUILD_TYPE: Release From aeb707c1d2543083f986eecf2dd09e863ef6db48 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 19 Jan 2022 14:54:20 -0800 Subject: [PATCH 030/277] Add CUDA tests for unsupported 4D data --- tests/src/endtoend/CMakeLists.txt | 2 +- tests/src/endtoend/cudaExecBase.c | 8 ++++++++ tests/src/endtoend/testZfpCuda4dDouble.c | 13 +++++++++++++ tests/src/endtoend/testZfpCuda4dFloat.c | 13 +++++++++++++ tests/src/endtoend/testZfpCuda4dInt32.c | 13 +++++++++++++ tests/src/endtoend/testZfpCuda4dInt64.c | 13 +++++++++++++ tests/src/endtoend/testcases/cuda.c | 5 +++++ 7 files changed, 66 insertions(+), 1 deletion(-) create mode 100644 tests/src/endtoend/testZfpCuda4dDouble.c create mode 100644 tests/src/endtoend/testZfpCuda4dFloat.c create mode 100644 tests/src/endtoend/testZfpCuda4dInt32.c create mode 100644 tests/src/endtoend/testZfpCuda4dInt64.c diff --git a/tests/src/endtoend/CMakeLists.txt b/tests/src/endtoend/CMakeLists.txt index 3d386c249..e3849f1ca 100644 --- a/tests/src/endtoend/CMakeLists.txt +++ b/tests/src/endtoend/CMakeLists.txt @@ -34,7 +34,7 @@ function(zfp_add_test dims type bits) endif() if(NOT DEFINED ZFP_OMP_TESTS_ONLY) - if(ZFP_WITH_CUDA AND (${dims} LESS 4)) + if(ZFP_WITH_CUDA) add_definitions(-DZFP_WITH_CUDA) set(cuda_test_name testZfpCuda${dims}d${type}) diff --git a/tests/src/endtoend/cudaExecBase.c b/tests/src/endtoend/cudaExecBase.c index 510ac199a..0e4042027 100644 --- a/tests/src/endtoend/cudaExecBase.c +++ b/tests/src/endtoend/cudaExecBase.c @@ -154,6 +154,14 @@ _catFunc3(given_, DESCRIPTOR, InterleavedArray_when_ZfpCompressFixedRate_expect_ runCompressDecompressNoopTest(state, zfp_mode_fixed_rate); } +#if DIMS == 4 +static void +_catFunc3(given_Cuda_, DIM_INT_STR, Array_when_ZfpCompressDecompress_expect_BitstreamUntouchedAndReturnsZero)(void **state) +{ + runCompressDecompressNoopTest(state, zfp_mode_fixed_rate); +} +#endif + /* setup functions */ static int diff --git a/tests/src/endtoend/testZfpCuda4dDouble.c b/tests/src/endtoend/testZfpCuda4dDouble.c new file mode 100644 index 000000000..afd05944f --- /dev/null +++ b/tests/src/endtoend/testZfpCuda4dDouble.c @@ -0,0 +1,13 @@ +#include "src/encode4d.c" + +#include "constants/4dDouble.h" +#include "cudaExecBase.c" + +int main() +{ + const struct CMUnitTest tests[] = { + #include "testcases/cuda.c" + }; + + return cmocka_run_group_tests(tests, setupRandomData, teardownRandomData); +} diff --git a/tests/src/endtoend/testZfpCuda4dFloat.c b/tests/src/endtoend/testZfpCuda4dFloat.c new file mode 100644 index 000000000..d0ce3ba89 --- /dev/null +++ b/tests/src/endtoend/testZfpCuda4dFloat.c @@ -0,0 +1,13 @@ +#include "src/encode4f.c" + +#include "constants/4dFloat.h" +#include "cudaExecBase.c" + +int main() +{ + const struct CMUnitTest tests[] = { + #include "testcases/cuda.c" + }; + + return cmocka_run_group_tests(tests, setupRandomData, teardownRandomData); +} diff --git a/tests/src/endtoend/testZfpCuda4dInt32.c b/tests/src/endtoend/testZfpCuda4dInt32.c new file mode 100644 index 000000000..bad538fb6 --- /dev/null +++ b/tests/src/endtoend/testZfpCuda4dInt32.c @@ -0,0 +1,13 @@ +#include "src/encode4i.c" + +#include "constants/4dInt32.h" +#include "cudaExecBase.c" + +int main() +{ + const struct CMUnitTest tests[] = { + #include "testcases/cuda.c" + }; + + return cmocka_run_group_tests(tests, setupRandomData, teardownRandomData); +} diff --git a/tests/src/endtoend/testZfpCuda4dInt64.c b/tests/src/endtoend/testZfpCuda4dInt64.c new file mode 100644 index 000000000..eb7c24d21 --- /dev/null +++ b/tests/src/endtoend/testZfpCuda4dInt64.c @@ -0,0 +1,13 @@ +#include "src/encode4l.c" + +#include "constants/4dInt64.h" +#include "cudaExecBase.c" + +int main() +{ + const struct CMUnitTest tests[] = { + #include "testcases/cuda.c" + }; + + return cmocka_run_group_tests(tests, setupRandomData, teardownRandomData); +} diff --git a/tests/src/endtoend/testcases/cuda.c b/tests/src/endtoend/testcases/cuda.c index a2e784eb5..0af0341f0 100644 --- a/tests/src/endtoend/testcases/cuda.c +++ b/tests/src/endtoend/testcases/cuda.c @@ -1,5 +1,6 @@ // requires #include "utils/testMacros.h", do outside of main() +#if DIMS < 4 _cmocka_unit_test(when_seededRandomSmoothDataGenerated_expect_ChecksumMatches), /* strided */ @@ -17,3 +18,7 @@ _cmocka_unit_test_setup_teardown(_catFunc3(given_Cuda_, DIM_INT_STR, Array_when_ /* non fixed-rate modes unsupported */ _cmocka_unit_test_setup_teardown(_catFunc3(given_Cuda_, DIM_INT_STR, Array_when_ZfpCompressDecompressNonFixedRate_expect_BitstreamUntouchedAndReturnsZero), setupDefaultStride, teardown), +#else +/* 4d compression unsupported */ +_cmocka_unit_test_setup_teardown(_catFunc3(given_Cuda_, DIM_INT_STR, Array_when_ZfpCompressDecompress_expect_BitstreamUntouchedAndReturnsZero), setupDefaultStride, teardown), +#endif From 38e3cbbe288ec0e0120d00bf3bfc2d472b43fe3f Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 19 Jan 2022 18:02:40 -0800 Subject: [PATCH 031/277] Silence const compiler warnings in ppm --- examples/ppm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/ppm.c b/examples/ppm.c index 1324ae0be..4b989a305 100644 --- a/examples/ppm.c +++ b/examples/ppm.c @@ -56,7 +56,7 @@ clamp(int32* block, uint n) /* convert 2D block from RGB to YCoCg color space */ static void -rgb2ycocg(int32 ycocg[3][16], const int32 rgb[3][16]) +rgb2ycocg(int32 ycocg[3][16], /*const*/ int32 rgb[3][16]) { uint i; for (i = 0; i < 16; i++) { @@ -80,7 +80,7 @@ rgb2ycocg(int32 ycocg[3][16], const int32 rgb[3][16]) /* convert 2D block from YCoCg to RGB color space */ static void -ycocg2rgb(int32 rgb[3][16], const int32 ycocg[3][16]) +ycocg2rgb(int32 rgb[3][16], /*const*/ int32 ycocg[3][16]) { uint i; for (i = 0; i < 16; i++) { From 4a33aab52ea7119cf06948e1a976f22079908fc3 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 20 Jan 2022 14:36:37 -0800 Subject: [PATCH 032/277] add missing zfp field and stream tests --- .../FindPythonExtensions.cmake | 1 - tests/src/misc/CMakeLists.txt | 32 ++ tests/src/misc/testZfpField1d.c | 24 ++ tests/src/misc/testZfpField1f.c | 24 ++ tests/src/misc/testZfpField2d.c | 28 ++ tests/src/misc/testZfpField2f.c | 28 ++ tests/src/misc/testZfpField3d.c | 32 ++ tests/src/misc/testZfpField3f.c | 32 ++ tests/src/misc/testZfpField4d.c | 36 ++ tests/src/misc/testZfpField4f.c | 36 ++ tests/src/misc/testZfpStream.c | 357 ++++++++++++++++++ tests/src/misc/zfpFieldBase.c | 257 +++++++++++++ 12 files changed, 886 insertions(+), 1 deletion(-) create mode 100644 tests/src/misc/testZfpField1d.c create mode 100644 tests/src/misc/testZfpField1f.c create mode 100644 tests/src/misc/testZfpField2d.c create mode 100644 tests/src/misc/testZfpField2f.c create mode 100644 tests/src/misc/testZfpField3d.c create mode 100644 tests/src/misc/testZfpField3f.c create mode 100644 tests/src/misc/testZfpField4d.c create mode 100644 tests/src/misc/testZfpField4f.c create mode 100644 tests/src/misc/zfpFieldBase.c diff --git a/python/scikit-build-cmake/FindPythonExtensions.cmake b/python/scikit-build-cmake/FindPythonExtensions.cmake index 78d82f5f4..33e034d09 100644 --- a/python/scikit-build-cmake/FindPythonExtensions.cmake +++ b/python/scikit-build-cmake/FindPythonExtensions.cmake @@ -331,7 +331,6 @@ function(_set_python_extension_symbol_visibility _target) else() set(_modinit_prefix "init") endif() - message("_modinit_prefix:${_modinit_prefix}") if("${CMAKE_C_COMPILER_ID}" STREQUAL "MSVC") set_target_properties(${_target} PROPERTIES LINK_FLAGS "/EXPORT:${_modinit_prefix}${_target}" diff --git a/tests/src/misc/CMakeLists.txt b/tests/src/misc/CMakeLists.txt index d659bab60..ce76cf0d8 100644 --- a/tests/src/misc/CMakeLists.txt +++ b/tests/src/misc/CMakeLists.txt @@ -10,6 +10,38 @@ add_executable(testZfpPromote testZfpPromote.c) target_link_libraries(testZfpPromote cmocka zfp) add_test(NAME testZfpPromote COMMAND testZfpPromote) +add_executable(testZfpField1f testZfpField1f.c) +target_link_libraries(testZfpField1f cmocka zfp) +add_test(NAME testZfpField1f COMMAND testZfpField1f) + +add_executable(testZfpField2f testZfpField2f.c) +target_link_libraries(testZfpField2f cmocka zfp) +add_test(NAME testZfpField2f COMMAND testZfpField2f) + +add_executable(testZfpField3f testZfpField3f.c) +target_link_libraries(testZfpField3f cmocka zfp) +add_test(NAME testZfpField3f COMMAND testZfpField3f) + +add_executable(testZfpField4f testZfpField4f.c) +target_link_libraries(testZfpField4f cmocka zfp) +add_test(NAME testZfpField4f COMMAND testZfpField4f) + +add_executable(testZfpField1d testZfpField1d.c) +target_link_libraries(testZfpField1d cmocka zfp) +add_test(NAME testZfpField1d COMMAND testZfpField1d) + +add_executable(testZfpField2d testZfpField2d.c) +target_link_libraries(testZfpField2d cmocka zfp) +add_test(NAME testZfpField2d COMMAND testZfpField2d) + +add_executable(testZfpField3d testZfpField3d.c) +target_link_libraries(testZfpField3d cmocka zfp) +add_test(NAME testZfpField3d COMMAND testZfpField3d) + +add_executable(testZfpField4d testZfpField4d.c) +target_link_libraries(testZfpField4d cmocka zfp) +add_test(NAME testZfpField4d COMMAND testZfpField4d) + if(HAVE_LIBM_MATH) target_link_libraries(testZfpHeader m) target_link_libraries(testZfpStream m) diff --git a/tests/src/misc/testZfpField1d.c b/tests/src/misc/testZfpField1d.c new file mode 100644 index 000000000..745e550f4 --- /dev/null +++ b/tests/src/misc/testZfpField1d.c @@ -0,0 +1,24 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 1 +#define ZFP_TYPE zfp_type_double +#define SCALAR double + +#define NX 20 +#define SX 2 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef SX diff --git a/tests/src/misc/testZfpField1f.c b/tests/src/misc/testZfpField1f.c new file mode 100644 index 000000000..4a416cdfb --- /dev/null +++ b/tests/src/misc/testZfpField1f.c @@ -0,0 +1,24 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 1 +#define ZFP_TYPE zfp_type_float +#define SCALAR float + +#define NX 20 +#define SX 2 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef SX diff --git a/tests/src/misc/testZfpField2d.c b/tests/src/misc/testZfpField2d.c new file mode 100644 index 000000000..7c48af609 --- /dev/null +++ b/tests/src/misc/testZfpField2d.c @@ -0,0 +1,28 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 2 +#define ZFP_TYPE zfp_type_double +#define SCALAR double + +#define NX 20 +#define NY 21 +#define SX 2 +#define SY 3 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef NY +#undef SX +#undef SY diff --git a/tests/src/misc/testZfpField2f.c b/tests/src/misc/testZfpField2f.c new file mode 100644 index 000000000..ebba9fead --- /dev/null +++ b/tests/src/misc/testZfpField2f.c @@ -0,0 +1,28 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 2 +#define ZFP_TYPE zfp_type_float +#define SCALAR float + +#define NX 20 +#define NY 21 +#define SX 2 +#define SY 3 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef NY +#undef SX +#undef SY diff --git a/tests/src/misc/testZfpField3d.c b/tests/src/misc/testZfpField3d.c new file mode 100644 index 000000000..c084276be --- /dev/null +++ b/tests/src/misc/testZfpField3d.c @@ -0,0 +1,32 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 3 +#define ZFP_TYPE zfp_type_double +#define SCALAR double + +#define NX 20 +#define NY 21 +#define NZ 12 +#define SX 2 +#define SY 3 +#define SZ 4 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef NY +#undef NZ +#undef SX +#undef SY +#undef SZ diff --git a/tests/src/misc/testZfpField3f.c b/tests/src/misc/testZfpField3f.c new file mode 100644 index 000000000..b4fca8265 --- /dev/null +++ b/tests/src/misc/testZfpField3f.c @@ -0,0 +1,32 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 3 +#define ZFP_TYPE zfp_type_float +#define SCALAR float + +#define NX 20 +#define NY 21 +#define NZ 12 +#define SX 2 +#define SY 3 +#define SZ 4 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef NY +#undef NZ +#undef SX +#undef SY +#undef SZ diff --git a/tests/src/misc/testZfpField4d.c b/tests/src/misc/testZfpField4d.c new file mode 100644 index 000000000..aeffbccb1 --- /dev/null +++ b/tests/src/misc/testZfpField4d.c @@ -0,0 +1,36 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 4 +#define ZFP_TYPE zfp_type_double +#define SCALAR double + +#define NX 20 +#define NY 21 +#define NZ 12 +#define NW 6 +#define SX 2 +#define SY 3 +#define SZ 4 +#define SW 2 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef NY +#undef NZ +#undef NW +#undef SX +#undef SY +#undef SZ +#undef SW diff --git a/tests/src/misc/testZfpField4f.c b/tests/src/misc/testZfpField4f.c new file mode 100644 index 000000000..9fdd60396 --- /dev/null +++ b/tests/src/misc/testZfpField4f.c @@ -0,0 +1,36 @@ +#include "zfp.h" + +#include +#include +#include +#include +#include + +#include + +#define DIMS 4 +#define ZFP_TYPE zfp_type_float +#define SCALAR float + +#define NX 20 +#define NY 21 +#define NZ 12 +#define NW 6 +#define SX 2 +#define SY 3 +#define SZ 4 +#define SW 2 + +#include "zfpFieldBase.c" + +#undef DIMS +#undef ZFP_TYPE +#undef SCALAR +#undef NX +#undef NY +#undef NZ +#undef NW +#undef SX +#undef SY +#undef SZ +#undef SW diff --git a/tests/src/misc/testZfpStream.c b/tests/src/misc/testZfpStream.c index e5d59211d..4f885a2ea 100644 --- a/tests/src/misc/testZfpStream.c +++ b/tests/src/misc/testZfpStream.c @@ -545,6 +545,343 @@ given_invalidCompressParamsModeVal_when_zfpStreamSetMode_expect_returnsNullMode_ assertCompressParamsBehaviorThroughSetMode(state, zfp_mode_null); } +static void +testStreamAlignSizeMatches(void **state, int dim, zfp_type type) +{ + struct setupVars *bundle = *state; + zfp_stream* stream = bundle->stream; + zfp_field* field; + + size_t arrsize = 4 << 2*(dim-1); + size_t dimsize = 4; + size_t flushsize; + size_t alignsize; + + if (type == zfp_type_float) + { + float* array; + float* block = (float*)calloc(arrsize, sizeof(float)); + + if (dim == 1) + { + array = (float*)calloc(dimsize, sizeof(float)); + field = zfp_field_1d(array, type, dimsize); + } + else if (dim == 2) + { + array = (float*)calloc(dimsize*dimsize, sizeof(float)); + field = zfp_field_2d(array, type, dimsize, dimsize); + } + else if (dim == 3) + { + array = (float*)calloc(dimsize*dimsize*dimsize, sizeof(float)); + field = zfp_field_3d(array, type, dimsize, dimsize, dimsize); + } + else if (dim == 4) + { + array = (float*)calloc(dimsize*dimsize*dimsize*dimsize, sizeof(float)); + field = zfp_field_4d(array, type, dimsize, dimsize, dimsize, dimsize); + } + + size_t bufsize = zfp_stream_maximum_size(stream, field); + void* buffer = malloc(bufsize); + bitstream* s = stream_open(buffer, bufsize); + zfp_stream_set_bit_stream(stream, s); + zfp_stream_rewind(stream); + + if (dim == 1) + { + zfp_encode_block_float_1(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_float_1(stream, block); + alignsize = zfp_stream_align(stream); + } + else if (dim == 2) + { + zfp_encode_block_float_2(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_float_2(stream, block); + alignsize = zfp_stream_align(stream); + } + else if (dim == 3) + { + zfp_encode_block_float_3(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_float_3(stream, block); + alignsize = zfp_stream_align(stream); + } + else if (dim == 4) + { + zfp_encode_block_float_4(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_float_4(stream, block); + alignsize = zfp_stream_align(stream); + } + + free(array); + free(block); + } + else if (type == zfp_type_double) + { + double* array; + double* block = (double*)calloc(arrsize, sizeof(double)); + + if (dim == 1) + { + array = (double*)calloc(dimsize, sizeof(double)); + field = zfp_field_1d(array, type, dimsize); + } + else if (dim == 2) + { + array = (double*)calloc(dimsize*dimsize, sizeof(double)); + field = zfp_field_2d(array, type, dimsize, dimsize); + } + else if (dim == 3) + { + array = (double*)calloc(dimsize*dimsize*dimsize, sizeof(double)); + field = zfp_field_3d(array, type, dimsize, dimsize, dimsize); + } + else if (dim == 4) + { + array = (double*)calloc(dimsize*dimsize*dimsize*dimsize, sizeof(double)); + field = zfp_field_4d(array, type, dimsize, dimsize, dimsize, dimsize); + } + + size_t bufsize = zfp_stream_maximum_size(stream, field); + void* buffer = malloc(bufsize); + bitstream* s = stream_open(buffer, bufsize); + zfp_stream_set_bit_stream(stream, s); + zfp_stream_rewind(stream); + + if (dim == 1) + { + zfp_encode_block_double_1(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_double_1(stream, block); + alignsize = zfp_stream_align(stream); + } + else if (dim == 2) + { + zfp_encode_block_double_2(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_double_2(stream, block); + alignsize = zfp_stream_align(stream); + } + else if (dim == 3) + { + zfp_encode_block_double_3(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_double_3(stream, block); + alignsize = zfp_stream_align(stream); + } + else if (dim == 4) + { + zfp_encode_block_double_4(stream, block); + flushsize = zfp_stream_flush(stream); + zfp_stream_rewind(stream); + zfp_decode_block_double_4(stream, block); + alignsize = zfp_stream_align(stream); + } + + free(array); + free(block); + } + + assert_true(flushsize > 0); + assert_true(flushsize == alignsize); +} + +static void +given_block1f_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 1, zfp_type_float); +} + +static void +given_block2f_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 2, zfp_type_float); +} + +static void +given_block3f_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 3, zfp_type_float); +} + +static void +given_block4f_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 4, zfp_type_float); +} + +static void +given_block1d_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 1, zfp_type_double); +} + +static void +given_block2d_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 2, zfp_type_double); +} + +static void +given_block3d_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 3, zfp_type_double); +} + +static void +given_block4d_when_StreamFlush_expect_StreamAlignSizeMatches(void **state) +{ + testStreamAlignSizeMatches(state, 4, zfp_type_double); +} + +static void +testStreamCompressedSizeIncreasedCorrectly(void **state, int dim, zfp_type type) +{ + struct setupVars *bundle = *state; + zfp_stream* stream = bundle->stream; + zfp_field* field; + + /* use fixed rate mode to simplify size calculation */ + double rate = zfp_stream_set_rate(stream, 64, type, dim, 0); + + size_t blocksize = 4 << 2*(dim-1); + size_t dimsize = 4; + size_t startsize; + size_t endsize; + + if (type == zfp_type_float) + { + float* array = (float*)calloc(blocksize, sizeof(float)); + float* block = (float*)calloc(blocksize, sizeof(float)); + + if (dim == 1) + field = zfp_field_1d(array, type, dimsize); + else if (dim == 2) + field = zfp_field_2d(array, type, dimsize, dimsize); + else if (dim == 3) + field = zfp_field_3d(array, type, dimsize, dimsize, dimsize); + else if (dim == 4) + field = zfp_field_4d(array, type, dimsize, dimsize, dimsize, dimsize); + + size_t bufsize = zfp_stream_maximum_size(stream, field); + void* buffer = malloc(bufsize); + bitstream* s = stream_open(buffer, bufsize); + zfp_stream_set_bit_stream(stream, s); + zfp_stream_rewind(stream); + startsize = zfp_stream_compressed_size(stream); + + if (dim == 1) + zfp_encode_block_float_1(stream, block); + else if (dim == 2) + zfp_encode_block_float_2(stream, block); + else if (dim == 3) + zfp_encode_block_float_3(stream, block); + else if (dim == 4) + zfp_encode_block_float_4(stream, block); + + endsize = zfp_stream_compressed_size(stream); + free(array); + free(block); + } + else if (type == zfp_type_double) + { + double* array = (double*)calloc(blocksize, sizeof(double)); + double* block = (double*)calloc(blocksize, sizeof(double)); + + if (dim == 1) + field = zfp_field_1d(array, type, dimsize); + else if (dim == 2) + field = zfp_field_2d(array, type, dimsize, dimsize); + else if (dim == 3) + field = zfp_field_3d(array, type, dimsize, dimsize, dimsize); + else if (dim == 4) + field = zfp_field_4d(array, type, dimsize, dimsize, dimsize, dimsize); + + size_t bufsize = zfp_stream_maximum_size(stream, field); + void* buffer = malloc(bufsize); + bitstream* s = stream_open(buffer, bufsize); + zfp_stream_set_bit_stream(stream, s); + zfp_stream_rewind(stream); + startsize = zfp_stream_compressed_size(stream); + + if (dim == 1) + zfp_encode_block_double_1(stream, block); + else if (dim == 2) + zfp_encode_block_double_2(stream, block); + else if (dim == 3) + zfp_encode_block_double_3(stream, block); + else if (dim == 4) + zfp_encode_block_double_4(stream, block); + + endsize = zfp_stream_compressed_size(stream); + free(array); + free(block); + } + + assert_true(endsize > 0); + assert_true(endsize == startsize + blocksize * (size_t)(rate/8)); +} + +static void +given_block1f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 1, zfp_type_float); +} + +static void +given_block2f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 2, zfp_type_float); +} + +static void +given_block3f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 3, zfp_type_float); +} + +static void +given_block4f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 4, zfp_type_float); +} + +static void +given_block1d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 1, zfp_type_double); +} + +static void +given_block2d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 2, zfp_type_double); +} + +static void +given_block3d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 3, zfp_type_double); +} + +static void +given_block4d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly(void **state) +{ + testStreamCompressedSizeIncreasedCorrectly(state, 4, zfp_type_double); +} + int main() { const struct CMUnitTest tests[] = { @@ -571,6 +908,26 @@ int main() cmocka_unit_test_setup_teardown(given_zfpStreamSetReversibleModeVal_when_zfpStreamSetMode_expect_returnsReversible_and_compressParamsConserved, setup, teardown), cmocka_unit_test_setup_teardown(given_customCompressParamsModeVal_when_zfpStreamSetMode_expect_returnsExpert_and_compressParamsConserved, setup, teardown), cmocka_unit_test_setup_teardown(given_invalidCompressParamsModeVal_when_zfpStreamSetMode_expect_returnsNullMode_and_paramsNotSet, setup, teardown), + + /* test other zfp_stream_align() */ + cmocka_unit_test_setup_teardown(given_block1f_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block2f_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block3f_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block4f_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block1d_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block2d_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block3d_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + cmocka_unit_test_setup_teardown(given_block4d_when_StreamFlush_expect_StreamAlignSizeMatches, setup, teardown), + + /* test zfp_stream_compressed_size() */ + cmocka_unit_test_setup_teardown(given_block1f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block2f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block3f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block4f_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block1d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block2d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block3d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), + cmocka_unit_test_setup_teardown(given_block4d_when_WriteBlock_expect_StreamCompressedSizeIncreasedCorrectly, setup, teardown), }; return cmocka_run_group_tests(tests, NULL, NULL); diff --git a/tests/src/misc/zfpFieldBase.c b/tests/src/misc/zfpFieldBase.c new file mode 100644 index 000000000..fcf2deda2 --- /dev/null +++ b/tests/src/misc/zfpFieldBase.c @@ -0,0 +1,257 @@ +struct setupVars { + zfp_field* field; + SCALAR* data; +}; + +static int +setupBasic(void **state) +{ + struct setupVars *bundle = malloc(sizeof(struct setupVars)); + assert_non_null(bundle); + +#if DIMS == 1 + zfp_field* field = zfp_field_1d(NULL, ZFP_TYPE, NX); +#elif DIMS == 2 + zfp_field* field = zfp_field_2d(NULL, ZFP_TYPE, NX, NY); +#elif DIMS == 3 + zfp_field* field = zfp_field_3d(NULL, ZFP_TYPE, NX, NY, NZ); +#elif DIMS == 4 + zfp_field* field = zfp_field_4d(NULL, ZFP_TYPE, NX, NY, NZ, NW); +#endif + + bundle->field = field; + bundle->data = NULL; + + *state = bundle; + + return 0; +} + +static int +setupContiguous(void **state) +{ + struct setupVars *bundle = malloc(sizeof(struct setupVars)); + assert_non_null(bundle); + +#if DIMS == 1 + zfp_field* field = zfp_field_1d(NULL, ZFP_TYPE, NX); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR)*NX); +#elif DIMS == 2 + zfp_field* field = zfp_field_2d(NULL, ZFP_TYPE, NX, NY); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR)*NX*NY); +#elif DIMS == 3 + zfp_field* field = zfp_field_3d(NULL, ZFP_TYPE, NX, NY, NZ); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR)*NX*NY*NZ); +#elif DIMS == 4 + zfp_field* field = zfp_field_4d(NULL, ZFP_TYPE, NX, NY, NZ, NW); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR)*NX*NY*NZ*NW); +#endif + assert_non_null(data); + + zfp_field_set_pointer(field, data); + bundle->field = field; + bundle->data = data; + + *state = bundle; + + return 0; +} + +static int +setupStrided(void **state) +{ + struct setupVars *bundle = malloc(sizeof(struct setupVars)); + assert_non_null(bundle); + +#if DIMS == 1 + zfp_field* field = zfp_field_1d(NULL, ZFP_TYPE, NX); + zfp_field_set_stride_1d(field, SX); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + 1)); +#elif DIMS == 2 + zfp_field* field = zfp_field_2d(NULL, ZFP_TYPE, NX, NY); + zfp_field_set_stride_2d(field, SX, SY); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + (SY*(NY-1)) + 1)); +#elif DIMS == 3 + zfp_field* field = zfp_field_3d(NULL, ZFP_TYPE, NX, NY, NZ); + zfp_field_set_stride_3d(field, SX, SY, SZ); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + (SY*(NY-1)) + (SZ*(NZ-1)) + 1)); +#elif DIMS == 4 + zfp_field* field = zfp_field_4d(NULL, ZFP_TYPE, NX, NY, NZ, NW); + zfp_field_set_stride_4d(field, SX, SY, SZ, SW); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + (SY*(NY-1)) + (SZ*(NZ-1)) + (SW*(NW-1)) + 1)); +#endif + assert_non_null(data); + + zfp_field_set_pointer(field, data); + bundle->field = field; + bundle->data = data; + + *state = bundle; + + return 0; +} + +static int +setupNegativeStrided(void **state) +{ + struct setupVars *bundle = malloc(sizeof(struct setupVars)); + assert_non_null(bundle); + +#if DIMS == 1 + zfp_field* field = zfp_field_1d(NULL, ZFP_TYPE, NX); + zfp_field_set_stride_1d(field, -SX); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + 1)); +#elif DIMS == 2 + zfp_field* field = zfp_field_2d(NULL, ZFP_TYPE, NX, NY); + zfp_field_set_stride_2d(field, -SX, -SY); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + (SY*(NY-1)) + 1)); +#elif DIMS == 3 + zfp_field* field = zfp_field_3d(NULL, ZFP_TYPE, NX, NY, NZ); + zfp_field_set_stride_3d(field, -SX, -SY, -SZ); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + (SY*(NY-1)) + (SZ*(NZ-1)) + 1)); +#elif DIMS == 4 + zfp_field* field = zfp_field_4d(NULL, ZFP_TYPE, NX, NY, NZ, NW); + zfp_field_set_stride_4d(field, -SX, -SY, -SZ, -SW); + SCALAR* data = (SCALAR*)malloc(sizeof(SCALAR) * ((SX*(NX-1)) + (SY*(NY-1)) + (SZ*(NZ-1)) + (SW*(NW-1)) + 1)); +#endif + assert_non_null(data); + + zfp_field_set_pointer(field, data); + bundle->field = field; + bundle->data = data; + + *state = bundle; + + return 0; +} + +static int +teardown(void **state) +{ + struct setupVars *bundle = *state; + + zfp_field_free(bundle->field); + + if (bundle->data != NULL) + free(bundle->data); + + free(bundle); + + return 0; +} + +static void +given_contiguousData_isContiguousReturnsTrue(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + + assert_true(zfp_field_is_contiguous(field)); +} + +static void +given_noncontiguousData_isContiguousReturnsFalse(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + + assert_false(zfp_field_is_contiguous(field)); +} + +static void +when_noFieldData_fieldBeginReturnsNull(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + + assert_null(zfp_field_begin(field)); +} + +static void +when_contiguousData_fieldBeginsAtDataPointer(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + + assert_true(zfp_field_begin(field) == zfp_field_pointer(field)); +} + +static void +when_noncontiguousDataWithNegativeStride_fieldBeginsAtCorrectLocation(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + SCALAR* data = bundle->data; + +#if DIMS == 1 + ptrdiff_t min = ((int)-SX * (ptrdiff_t)(NX - 1)); +#elif DIMS == 2 + ptrdiff_t min = ((int)-SX * (ptrdiff_t)(NX - 1)) + ((int)-SY * (ptrdiff_t)(NY - 1)); +#elif DIMS == 3 + ptrdiff_t min = ((int)-SX * (ptrdiff_t)(NX - 1)) + ((int)-SY * (ptrdiff_t)(NY - 1)) + ((int)-SZ * (ptrdiff_t)(NZ - 1)); +#elif DIMS == 4 + ptrdiff_t min = ((int)-SX * (ptrdiff_t)(NX - 1)) + ((int)-SY * (ptrdiff_t)(NY - 1)) + ((int)-SZ * (ptrdiff_t)(NZ - 1)) + ((int)-SW * (ptrdiff_t)(NW - 1)); +#endif + void* begin = (void*)((uchar*)field->data + min * (ptrdiff_t)zfp_type_size(field->type)); + assert_true(zfp_field_begin(field) == begin); +} + +static void +given_field_precisionCorrect(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + + assert_true(zfp_field_precision(field) == sizeof(SCALAR) * CHAR_BIT); +} + +static void +given_contiguousData_fieldSizeBytesCorrect(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + +#if DIMS == 1 + assert_true(zfp_field_size_bytes(field) == NX * sizeof(SCALAR)); +#elif DIMS == 2 + assert_true(zfp_field_size_bytes(field) == NX * NY * sizeof(SCALAR)); +#elif DIMS == 3 + assert_true(zfp_field_size_bytes(field) == NX * NY * NZ * sizeof(SCALAR)); +#elif DIMS == 4 + assert_true(zfp_field_size_bytes(field) == NX * NY * NZ * NW * sizeof(SCALAR)); +#endif +} + +static void +given_noncontiguousData_fieldSizeBytesCorrect(void **state) +{ + struct setupVars *bundle = *state; + zfp_field* field = bundle->field; + +#if DIMS == 1 + assert_true(zfp_field_size_bytes(field) == ((SX*(NX-1) + 1) * sizeof(SCALAR))); +#elif DIMS == 2 + assert_true(zfp_field_size_bytes(field) == ((SX*(NX-1) + SY*(NY-1) + 1) * sizeof(SCALAR))); +#elif DIMS == 3 + assert_true(zfp_field_size_bytes(field) == ((SX*(NX-1) + SY*(NY-1) + SZ*(NZ-1) + 1) * sizeof(SCALAR))); +#elif DIMS == 4 + assert_true(zfp_field_size_bytes(field) == ((SX*(NX-1) + SY*(NY-1) + SZ*(NZ-1) + SW*(NW-1) + 1) * sizeof(SCALAR))); +#endif +} + + + +int main() +{ + const struct CMUnitTest tests[] = { + cmocka_unit_test_setup_teardown(given_contiguousData_isContiguousReturnsTrue, setupContiguous, teardown), + cmocka_unit_test_setup_teardown(given_noncontiguousData_isContiguousReturnsFalse, setupStrided, teardown), + cmocka_unit_test_setup_teardown(when_noFieldData_fieldBeginReturnsNull, setupBasic, teardown), + cmocka_unit_test_setup_teardown(when_contiguousData_fieldBeginsAtDataPointer, setupContiguous, teardown), + cmocka_unit_test_setup_teardown(when_noncontiguousDataWithNegativeStride_fieldBeginsAtCorrectLocation, setupNegativeStrided, teardown), + cmocka_unit_test_setup_teardown(given_field_precisionCorrect, setupBasic, teardown), + cmocka_unit_test_setup_teardown(given_contiguousData_fieldSizeBytesCorrect, setupContiguous, teardown), + cmocka_unit_test_setup_teardown(given_noncontiguousData_fieldSizeBytesCorrect, setupStrided, teardown), + }; + return cmocka_run_group_tests(tests, NULL, NULL); +} From 2b03babb826faa0f57a82eacf8362b2b3e96fb6c Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Feb 2022 07:37:40 -0800 Subject: [PATCH 033/277] Add conda zfp package and zarr application to docs --- docs/source/introduction.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index 90a90ba0e..d0e110430 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -55,7 +55,8 @@ source code is recommended for users who wish to configure the internals of bindings) to install. |zfp| is also available through several package managers, including -`Conda `__, +Conda (both `C/C++ `__ and +`Python `__ packages are available), `PIP `__, and `Spack `__. `RPM packages `__ are available @@ -99,6 +100,8 @@ plugins, and formats, such as * `Compression worklet `__ in `VTK-m `__. +* `Compression codec `__ in `Zarr `__ via `numcodecs `__. + See `this list `__ for other software products that support |zfp|. From f4e0850c674ae6a3a6bf9b211ed43623a6b8a977 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Feb 2022 07:38:32 -0800 Subject: [PATCH 034/277] Clarify that rounding mode, daz supported only on CPU --- docs/source/installation.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 852bde92c..416f454ae 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -275,6 +275,8 @@ in the same manner that :ref:`build targets ` are specified, e.g., libraries built with the default rounding mode, :code:`ZFP_ROUND_NEVER`, and versions of |zfp| that do not support a rounding mode with no adverse effects. + Note: :c:macro:`ZFP_ROUNDING_MODE` is currently supported only by the + :code:`serial` and :code:`omp` :ref:`execution policies `. Default: :code:`ZFP_ROUND_NEVER`. .. c:macro:: ZFP_WITH_TIGHT_ERROR @@ -286,7 +288,9 @@ in the same manner that :ref:`build targets ` are specified, e.g., to be satisfied using fewer bits of compressed data. As a result, when enabled, the observed maximum absolute error is closer to the tolerance and the compression ratio is increased. This feature requires the rounding mode - to be :code:`ZFP_ROUND_FIRST` or :code:`ZFP_ROUND_LAST`. + to be :code:`ZFP_ROUND_FIRST` or :code:`ZFP_ROUND_LAST` and is supported + only by the :code:`serial` and :code:`omp` + :ref:`execution policies `. Default: undefined/off. .. c:macro:: ZFP_WITH_DAZ @@ -303,6 +307,9 @@ in the same manner that :ref:`build targets ` are specified, e.g., results in "random" subnormals upon decompression. When enabled, compressed streams may differ slightly but are decompressed correctly by libraries built without this option. This option may break some regression tests. + Note: :c:macro:`ZFP_WITH_DAZ` is currently ignored by all + :ref:`execution policies ` other than :code:`serial` and + :code:`omp`. Default: undefined/off. .. c:macro:: ZFP_WITH_ALIGNED_ALLOC From 33065d2375e78a9aced7a7a257c829245a7468f1 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Feb 2022 07:50:42 -0800 Subject: [PATCH 035/277] Add support for generic codec in diffusion example --- CMakeLists.txt | 12 +- docs/source/examples.rst | 45 ++++--- examples/CMakeLists.txt | 7 +- examples/diffusion.cpp | 277 +++++++++++++++++++++++++++------------ 4 files changed, 235 insertions(+), 106 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 9652a23f9..24ba7d657 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -162,10 +162,18 @@ if(DEFINED ZFP_WITH_OPENMP) option(ZFP_WITH_OPENMP "Enable OpenMP parallel compression" ${ZFP_WITH_OPENMP}) if(ZFP_WITH_OPENMP) - find_package(OpenMP COMPONENTS C REQUIRED) + if(BUILD_EXAMPLES) + find_package(OpenMP COMPONENTS C CXX REQUIRED) + else() + find_package(OpenMP COMPONENTS C REQUIRED) + endif() endif() else() - find_package(OpenMP COMPONENTS C) + if(BUILD_EXAMPLES) + find_package(OpenMP COMPONENTS C CXX) + else() + find_package(OpenMP COMPONENTS C) + endif() option(ZFP_WITH_OPENMP "Enable OpenMP parallel compression" ${OPENMP_FOUND}) endif() diff --git a/docs/source/examples.rst b/docs/source/examples.rst index 14feeedde..a6ad92a98 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -39,8 +39,11 @@ solution and solution time. The usage is:: -a : absolute error tolerance (requires -c) -b : cache size in number of 4x4 blocks -c : use read-only arrays (needed for -a, -p, -R) + -d : use double-precision tiled arrays + -f : use single-precision tiled arrays + -h : use half-precision tiled arrays -i : traverse arrays using iterators instead of integer indices - -j : use OpenMP parallel execution + -j : use OpenMP parallel execution (requires -r) -n : grid dimensions -p : precision in uncompressed bits/value (requires -c) -r : rate in compressed bits/value @@ -48,12 +51,11 @@ solution and solution time. The usage is:: -t : number of time steps Here *rate* specifies the exact number of compressed bits to store per -double-precision floating-point value (default = 64); *nx* and *ny* -specify the grid size (default = 100 |times| 100); *nt* specifies the number -of time steps to take (the default is to run until time *t* = 1); and *blocks* -is the number of uncompressed blocks to cache -(default = (|sqrt| *n*) / 4, where *n* = *nx* |times| *ny*). -The :code:`-i` option enables array traversal via iterators instead of indices. +double-precision floating-point value; *nx* and *ny* specify the grid size +(default = 128 |times| 128); *nt* specifies the number of time steps to take +(the default is to run until time *t* = 1); and *blocks* is the number of +uncompressed blocks to cache (default = *nx* / 2). The :code:`-i` option +enables array traversal via iterators instead of indices. The :code:`-j` option enables OpenMP parallel execution, which makes use of both mutable and immutable :ref:`private views ` @@ -66,22 +68,33 @@ This example also illustrates how :ref:`read-only arrays ` fixed-precision (:code:`-p`), fixed-accuracy (:code:`-a`), or reversible (:code:`-R`) mode. -Running diffusion with the following arguments:: +The output lists for each time step the current rate of the state array and +in parentheses any additional storage, e.g., for the block +:ref:`cache ` and :ref:`index ` data structures, both in bits +per array element. Running diffusion with the following arguments:: diffusion -r 8 diffusion -r 12 - diffusion -r 20 - diffusion -r 64 + diffusion -r 16 + diffusion -r 24 + diffusion -should result in this output:: +should result in this final output:: - sum=0.996442 error=4.813938e-07 - sum=0.998338 error=1.967777e-07 - sum=0.998326 error=1.967952e-07 - sum=0.998326 error=1.967957e-07 + sum=0.995170 error=4.044954e-07 + sum=0.998151 error=1.237837e-07 + sum=0.998345 error=1.212734e-07 + sum=0.998346 error=1.212716e-07 + sum=0.998346 error=1.212716e-07 For speed and quality comparison, the solver solves the same problem using -uncompressed double-precision arrays when compression parameters are omitted. +uncompressed double-precision row-major arrays when compression parameters +are omitted. If one of :code:`-h`, :code:`-f`, :code:`-d` is specified, +uncompressed tiled arrays are used. These arrays are based on the |zfp| +array classes but make use of the :ref:`generic codec `, which +stores blocks as uncompressed scalars of the specified type (:code:`half`, +:code:`float`, or :code:`double`) while utilizing a double-precision block +cache (like |zfp|'s compressed arrays). The :program:`diffusionC` program is the same example written entirely in C using the |cfp| :ref:`wrappers ` around the C++ compressed array diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt index 565e929a1..2861e470a 100644 --- a/examples/CMakeLists.txt +++ b/examples/CMakeLists.txt @@ -1,6 +1,11 @@ add_executable(diffusion diffusion.cpp) -target_link_libraries(diffusion zfp) target_compile_definitions(diffusion PRIVATE ${zfp_compressed_array_defs}) +if(ZFP_WITH_OPENMP) + target_link_libraries(diffusion zfp OpenMP::OpenMP_CXX) +else() + target_link_libraries(diffusion zfp) +endif() + if(BUILD_CFP) add_executable(diffusionC diffusionC.c) diff --git a/examples/diffusion.cpp b/examples/diffusion.cpp index 14fe0998c..7b8bc150f 100644 --- a/examples/diffusion.cpp +++ b/examples/diffusion.cpp @@ -8,12 +8,39 @@ #include #include "zfparray2.h" #include "zfpcarray2.h" +#include "gencodec.h" #include "array2d.h" +// add half precision if compiler supports it +#define __STDC_WANT_IEC_60559_TYPES_EXT__ +#include +#ifdef FLT16_MAX + #define WITH_HALF 1 +#else + #undef WITH_HALF +#endif + #ifdef _OPENMP #include #endif +// uncompressed tiled arrays based on zfp generic codec +namespace tiled { +#if WITH_HALF + typedef zfp::array2< double, zfp::codec::generic2 > array2h; +#endif + typedef zfp::array2< double, zfp::codec::generic2 > array2f; + typedef zfp::array2< double, zfp::codec::generic2 > array2d; +} + +// enumeration of uncompressed storage types +enum storage_type { + type_none = 0, + type_half = 1, + type_float = 2, + type_double = 3 +}; + // constants used in the solution class Constants { public: @@ -54,20 +81,22 @@ laplacian(const array2d& u, size_t x, size_t y, const Constants& c) return uxx + uyy; } -template +template inline void -time_step_parallel(array2d& u, const Constants& c); +time_step_parallel(state& u, scratch& v, const Constants& c); -// advance solution in parallel via thread-safe views #ifdef _OPENMP +// advance solution in parallel via thread-safe views template <> inline void -time_step_parallel(zfp::array2d& u, const Constants& c) +time_step_parallel(zfp::array2d& u, zfp::array2d& du, const Constants& c) { +#if 0 // flush shared cache to ensure cache consistency across threads u.flush_cache(); + // zero-initialize du + du.set(0); // compute du/dt in parallel - zfp::array2d du(c.nx, c.ny, u.rate(), 0, u.cache_size()); #pragma omp parallel { // create read-only private view of entire array u @@ -75,21 +104,45 @@ time_step_parallel(zfp::array2d& u, const Constants& c) // create read-write private view into rectangular subset of du zfp::array2d::private_view mydu(&du); mydu.partition(omp_get_thread_num(), omp_get_num_threads()); +//fprintf(stderr, "thread=%d (%zu, %zu) (%zu, %zu)\n", omp_get_thread_num(), mydu.global_x(0), mydu.global_y(0), mydu.size_x(), mydu.size_y()); // process rectangular region owned by this thread for (size_t j = 0; j < mydu.size_y(); j++) { size_t y = mydu.global_y(j); if (1 <= y && y <= c.ny - 2) for (size_t i = 0; i < mydu.size_x(); i++) { size_t x = mydu.global_x(i); - if (1 <= x && x <= c.nx - 2) { -#if 0 - double uxx = (myu(x - 1, y) - 2 * myu(x, y) + myu(x + 1, y)) / (c.dx * c.dx); - double uyy = (myu(x, y - 1) - 2 * myu(x, y) + myu(x, y + 1)) / (c.dy * c.dy); - mydu(i, j) = c.dt * c.k * (uxx + uyy); + if (1 <= x && x <= c.nx - 2) + mydu(i, j) = c.dt * c.k * laplacian(myu, x, y, c); + } + } + // compress all private cached blocks to shared storage + mydu.flush_cache(); + } + // take forward Euler step in serial + for (size_t i = 0; i < u.size(); i++) + u[i] += du[i]; #else + // flush shared cache to ensure cache consistency across threads + u.flush_cache(); + // zero-initialize du + du.set(0); + // compute du/dt in parallel + uint omp_num_threads = 8; + for (uint omp_thread_num = 0; omp_thread_num < omp_num_threads; omp_thread_num++) + { + // create read-only private view of entire array u + zfp::array2d::private_const_view myu(&u); + // create read-write private view into rectangular subset of du + zfp::array2d::private_view mydu(&du); + mydu.partition(omp_thread_num, omp_num_threads); + // process rectangular region owned by this thread + for (size_t j = 0; j < mydu.size_y(); j++) { + size_t y = mydu.global_y(j); + if (1 <= y && y <= c.ny - 2) + for (size_t i = 0; i < mydu.size_x(); i++) { + size_t x = mydu.global_x(i); + if (1 <= x && x <= c.nx - 2) mydu(i, j) = c.dt * c.k * laplacian(myu, x, y, c); -#endif - } } } // compress all private cached blocks to shared storage @@ -98,36 +151,34 @@ time_step_parallel(zfp::array2d& u, const Constants& c) // take forward Euler step in serial for (size_t i = 0; i < u.size(); i++) u[i] += du[i]; +#endif } #else +// dummy template instantiation when OpenMP support is not available template <> -inline void -time_step_parallel(zfp::array2d&, const Constants&) -{ -} +inline void time_step_parallel(zfp::array2d&, zfp::array2d&, const Constants&) {} #endif -// dummy template instantiation; never executed +// dummy template instantiations; never executed template <> -inline void -time_step_parallel(zfp::const_array2d&, const Constants&) -{ -} - -// dummy template instantiation; never executed +inline void time_step_parallel(zfp::const_array2d&, raw::array2d&, const Constants&) {} template <> -inline void -time_step_parallel(raw::array2d&, const Constants&) -{ -} +inline void time_step_parallel(raw::array2d&, raw::array2d&, const Constants&) {} +template <> +inline void time_step_parallel(tiled::array2d&, tiled::array2d&, const Constants&) {} +template <> +inline void time_step_parallel(tiled::array2f&, tiled::array2f&, const Constants&) {} +#if WITH_HALF +template <> +inline void time_step_parallel(tiled::array2h&, tiled::array2h&, const Constants&) {} +#endif // advance solution using integer array indices (generic implementation) -template +template inline void -time_step_indexed(array2d& u, const Constants& c) +time_step_indexed(state& u, scratch& du, const Constants& c) { // compute du/dt - array2d du(c.nx, c.ny, u.rate(), 0, u.cache_size()); for (size_t y = 1; y < c.ny - 1; y++) for (size_t x = 1; x < c.nx - 1; x++) du(x, y) = c.dt * c.k * laplacian(u, x, y, c); @@ -139,10 +190,9 @@ time_step_indexed(array2d& u, const Constants& c) // advance solution using integer array indices (read-only arrays) template <> inline void -time_step_indexed(zfp::const_array2d& u, const Constants& c) +time_step_indexed(zfp::const_array2d& u, raw::array2d& v, const Constants& c) { // initialize v as uncompressed copy of u - raw::array2d v(c.nx, c.ny); u.get(&v[0]); // take forward Euler step v += (du/dt) dt for (size_t y = 1; y < c.ny - 1; y++) @@ -153,48 +203,46 @@ time_step_indexed(zfp::const_array2d& u, const Constants& c) } // advance solution using array iterators (generic implementation) -template +template inline void -time_step_iterated(array2d& u, const Constants& c) +time_step_iterated(state& u, scratch& du, const Constants& c) { // compute du/dt - array2d du(c.nx, c.ny, u.rate(), 0, u.cache_size()); - for (typename array2d::iterator p = du.begin(); p != du.end(); p++) { - size_t x = p.i(); - size_t y = p.j(); + for (typename scratch::iterator q = du.begin(); q != du.end(); q++) { + size_t x = q.i(); + size_t y = q.j(); if (1 <= x && x <= c.nx - 2 && 1 <= y && y <= c.ny - 2) - *p = c.dt * c.k * laplacian(u, x, y, c); + *q = c.dt * c.k * laplacian(u, x, y, c); } // take forward Euler step - for (typename array2d::iterator p = u.begin(), q = du.begin(); p != u.end(); p++, q++) - *p += *q; + for (typename state::iterator p = u.begin(); p != u.end(); p++) + *p += du(p.i(), p.j()); } -// dummy specialization; never called +// advance solution using array iterators (read-only arrays) template <> inline void -time_step_iterated(zfp::const_array2d& u, const Constants& c) +time_step_iterated(zfp::const_array2d& u, raw::array2d& v, const Constants& c) { // initialize v as uncompressed copy of u - raw::array2d v(c.nx, c.ny); u.get(&v[0]); // take forward Euler step v += (du/dt) dt - for (raw::array2d::iterator p = v.begin(); p != v.end(); p++) { - size_t x = p.i(); - size_t y = p.j(); + for (raw::array2d::iterator q = v.begin(); q != v.end(); q++) { + size_t x = q.i(); + size_t y = q.j(); if (1 <= x && x <= c.nx - 2 && 1 <= y && y <= c.ny - 2) - *p += c.dt * c.k * laplacian(u, x, y, c); + *q += c.dt * c.k * laplacian(u, x, y, c); } // update u with uncompressed copy v u.set(&v[0]); } // set initial conditions with a point heat source (u is assumed zero-initialized) -template +template inline void -initialize(array2d& u, const Constants& c) +initialize(state& u, scratch&, const Constants& c) { u(c.x0, c.y0) = 1; } @@ -202,20 +250,19 @@ initialize(array2d& u, const Constants& c) // set initial conditions for const_array; requires updating the whole array template <> inline void -initialize(zfp::const_array2d& u, const Constants& c) +initialize(zfp::const_array2d& u, raw::array2d& v, const Constants& c) { - std::vector data(c.nx * c.ny, 0.0); - data[c.x0 + c.nx * c.y0] = 1; - u.set(&data[0]); + v(c.x0, c.y0) = 1; + u.set(&v[0]); } // solve heat equation -template +template inline double -solve(array2d& u, const Constants& c, bool iterator, bool parallel) +solve(state& u, scratch& v, const Constants& c, bool iterator, bool parallel) { // initialize u with point heat source - initialize(u, c); + initialize(u, v, c); // iterate until final time double t; @@ -227,20 +274,20 @@ solve(array2d& u, const Constants& c, bool iterator, bool parallel) std::cerr << "rate=" << std::setprecision(3) << std::fixed << rate << " (+" << rest << ")" << std::endl; // advance solution one time step if (parallel) - time_step_parallel(u, c); + time_step_parallel(u, v, c); else if (iterator) - time_step_iterated(u, c); + time_step_iterated(u, v, c); else - time_step_indexed(u, c); + time_step_indexed(u, v, c); } return t; } // compute sum of array values -template +template inline double -total(const array2d& u) +total(const state& u) { double s = 0; const size_t nx = u.size_x(); @@ -252,9 +299,9 @@ total(const array2d& u) } // compute root mean square error with respect to exact solution -template +template inline double -error(const array2d& u, const Constants& c, double t) +error(const state& u, const Constants& c, double t) { double e = 0; for (size_t y = 1; y < c.ny - 1; y++) { @@ -269,6 +316,20 @@ error(const array2d& u, const Constants& c, double t) return std::sqrt(e / ((c.nx - 2) * (c.ny - 2))); } +// execute solver and evaluate error +template +inline void +execute(state& u, scratch& v, size_t nt, bool iterator, bool parallel) +{ + Constants c(u.size_x(), u.size_y(), nt); + double t = solve(u, v, c, iterator, parallel); + double sum = total(u); + double err = error(u, c, t); + std::cerr.unsetf(std::ios::fixed); + std::cerr << "sum=" << std::setprecision(6) << std::fixed << sum << " error=" << std::setprecision(6) << std::scientific << err << std::endl; +} + +// print usage information inline int usage() { @@ -276,7 +337,12 @@ usage() std::cerr << "Options:" << std::endl; std::cerr << "-a : use compressed arrays with given absolute error tolerance" << std::endl; std::cerr << "-b : use 'blocks' 4x4 blocks of cache" << std::endl; - std::cerr << "-c : use read-only arrays" << std::endl; + std::cerr << "-c : use read-only compressed arrays" << std::endl; + std::cerr << "-d : use double-precision tiled arrays" << std::endl; + std::cerr << "-f : use single-precision tiled arrays" << std::endl; +#if WITH_HALF + std::cerr << "-h : use half-precision tiled arrays" << std::endl; +#endif std::cerr << "-i : traverse arrays using iterators" << std::endl; #ifdef _OPENMP std::cerr << "-j : use multithreading (only with compressed arrays)" << std::endl; @@ -291,15 +357,15 @@ usage() int main(int argc, char* argv[]) { - size_t nx = 100; - size_t ny = 100; + size_t nx = 128; + size_t ny = 128; size_t nt = 0; - double rate = 64; size_t cache_size = 0; zfp_config config = zfp_config_none(); bool iterator = false; bool parallel = false; bool writable = true; + storage_type type = type_none; // parse command-line options for (int i = 1; i < argc; i++) @@ -316,6 +382,14 @@ int main(int argc, char* argv[]) } else if (std::string(argv[i]) == "-c") writable = false; + else if (std::string(argv[i]) == "-d") + type = type_double; + else if (std::string(argv[i]) == "-f") + type = type_float; +#if WITH_HALF + else if (std::string(argv[i]) == "-h") + type = type_half; +#endif else if (std::string(argv[i]) == "-i") iterator = true; #ifdef _OPENMP @@ -334,6 +408,7 @@ int main(int argc, char* argv[]) config = zfp_config_precision(precision); } else if (std::string(argv[i]) == "-r") { + double rate; if (++i == argc || sscanf(argv[i], "%lf", &rate) != 1) return usage(); config = zfp_config_rate(rate, false); @@ -370,38 +445,66 @@ int main(int argc, char* argv[]) fprintf(stderr, "read-only arrays require compression parameters\n"); return EXIT_FAILURE; } + if (compression && type != type_none) { + fprintf(stderr, "tiled arrays do not support compression parameters\n"); + return EXIT_FAILURE; + } - Constants c(nx, ny, nt); + // if unspecified, set cache size to two layers of blocks + if (!cache_size) + cache_size = 2 * 4 * nx * sizeof(double); - double sum; - double err; + // solve problem if (compression) { - // solve problem using compressed arrays + // use compressed arrays if (writable) { // use read-write fixed-rate arrays - zfp::array2d u(nx, ny, rate, 0, cache_size); - double t = solve(u, c, iterator, parallel); - sum = total(u); - err = error(u, c, t); + zfp::array2d u(nx, ny, config.arg.rate, 0, cache_size); + zfp::array2d v(nx, ny, config.arg.rate, 0, cache_size); + execute(u, v, nt, iterator, parallel); } else { // use read-only variable-rate arrays zfp::const_array2d u(nx, ny, config, 0, cache_size); - double t = solve(u, c, iterator, parallel); - sum = total(u); - err = error(u, c, t); + raw::array2d v(nx, ny); + execute(u, v, nt, iterator, parallel); } } else { - // solve problem using uncompressed arrays - raw::array2d u(nx, ny); - double t = solve(u, c, iterator, parallel); - sum = total(u); - err = error(u, c, t); + // use uncompressed arrays + switch (type) { +#if WITH_HALF + case type_half: { + // use zfp generic codec with tiled half-precision storage + tiled::array2h u(nx, ny, sizeof(__fp16) * CHAR_BIT, 0, cache_size); + tiled::array2h v(nx, ny, sizeof(__fp16) * CHAR_BIT, 0, cache_size); + execute(u, v, nt, iterator, parallel); + } + break; +#endif + case type_float: { + // use zfp generic codec with tiled single-precision storage + tiled::array2f u(nx, ny, sizeof(float) * CHAR_BIT, 0, cache_size); + tiled::array2f v(nx, ny, sizeof(float) * CHAR_BIT, 0, cache_size); + execute(u, v, nt, iterator, parallel); + } + break; + case type_double: { + // use zfp generic codec with tiled double-precision storage + tiled::array2d u(nx, ny, sizeof(double) * CHAR_BIT, 0, cache_size); + tiled::array2d v(nx, ny, sizeof(double) * CHAR_BIT, 0, cache_size); + execute(u, v, nt, iterator, parallel); + } + break; + default: { + // use uncompressed array with row-major double-precision storage + raw::array2d u(nx, ny, sizeof(double) * CHAR_BIT); + raw::array2d v(nx, ny, sizeof(double) * CHAR_BIT); + execute(u, v, nt, iterator, parallel); + } + break; + } } - std::cerr.unsetf(std::ios::fixed); - std::cerr << "sum=" << std::setprecision(6) << std::fixed << sum << " error=" << std::setprecision(6) << std::scientific << err << std::endl; - return 0; } From da4e282936048fdfbae15ff56494c0576c53c88e Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Feb 2022 08:14:11 -0800 Subject: [PATCH 036/277] Allow setting BIT_STREAM_WORD_TYPE on make line --- Config | 55 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/Config b/Config index a2d7aba5f..107a56d33 100644 --- a/Config +++ b/Config @@ -20,21 +20,33 @@ SOFLAGS = # OpenMP compiler options ----------------------------------------------------- -# do not uncomment; use "make ZFP_WITH_OPENMP=0" to disable OpenMP +# do not comment out; use "make ZFP_WITH_OPENMP=0" to disable OpenMP OMPFLAGS = -fopenmp # optional compiler macros ---------------------------------------------------- -# use long long for 64-bit types -# DEFS += -DZFP_INT64='long long' -DZFP_INT64_SUFFIX='ll' -# DEFS += -DZFP_UINT64='unsigned long long' -DZFP_UINT64_SUFFIX='ull' - -# use smaller bit stream word type for finer rate granularity +# use smaller bit stream word type for finer rate granularity; +# can bet set on command line, e.g., "make BIT_STREAM_WORD_TYPE=uint8" # DEFS += -DBIT_STREAM_WORD_TYPE=uint8 # DEFS += -DBIT_STREAM_WORD_TYPE=uint16 # DEFS += -DBIT_STREAM_WORD_TYPE=uint32 # DEFS += -DBIT_STREAM_WORD_TYPE=uint64 +# reduce bias and slack in errors; can be set on command line, e.g., +# "make ZFP_ROUNDING_MODE=ZFP_ROUND_FIRST" +# DEFS += -DZFP_ROUNDING_MODE=ZFP_ROUND_NEVER +# DEFS += -DZFP_ROUNDING_MODE=ZFP_ROUND_FIRST +# DEFS += -DZFP_ROUNDING_MODE=ZFP_ROUND_LAST +# DEFS += -DZFP_WITH_TIGHT_ERROR + +# treat subnormals as zero to avoid overflow; can be set on command line, e.g., +# "make ZFP_WITH_DAZ=1" +# DEFS += -DZFP_WITH_DAZ + +# use long long for 64-bit types +# DEFS += -DZFP_INT64='long long' -DZFP_INT64_SUFFIX='ll' +# DEFS += -DZFP_UINT64='unsigned long long' -DZFP_UINT64_SUFFIX='ull' + # cache alignment # DEFS += -DZFP_CACHE_LINE_SIZE=256 @@ -53,14 +65,6 @@ OMPFLAGS = -fopenmp # count cache misses # DEFS += -DZFP_WITH_CACHE_PROFILE -# reduce bias and slack in errors -# DEFS += -DZFP_ROUNDING_MODE=ZFP_ROUND_FIRST -# DEFS += -DZFP_ROUNDING_MODE=ZFP_ROUND_LAST -# DEFS += -DZFP_WITH_TIGHT_ERROR - -# treat subnormals as zero to avoid overflow -# DEFS += -DZFP_WITH_DAZ - # build targets --------------------------------------------------------------- # default targets @@ -93,7 +97,7 @@ else LIBCFP = libcfp.a endif -# conditionals ---------------------------------------------------------------- +# operating system and compiler dependent flags ------------------------------- # macOS configuration; compile with "make OS=mac" ifeq ($(OS),mac) @@ -105,6 +109,13 @@ ifeq ($(CSTD),-std=c89) FLAGS += -Wno-unused-function endif +# process macros set on the command line -------------------------------------- + +# bit stream word type +ifdef BIT_STREAM_WORD_TYPE + DEFS += -DBIT_STREAM_WORD_TYPE=$(BIT_STREAM_WORD_TYPE) +endif + # enable OpenMP? ifdef ZFP_WITH_OPENMP ifneq ($(ZFP_WITH_OPENMP),0) @@ -114,6 +125,13 @@ ifdef ZFP_WITH_OPENMP endif endif +# treat subnormals as zero to avoid overflow +ifdef ZFP_WITH_DAZ + ifneq ($(ZFP_WITH_DAZ),0) + FLAGS += -DZFP_WITH_DAZ + endif +endif + # rounding mode and slack in error ifdef ZFP_ROUNDING_MODE FLAGS += -DZFP_ROUNDING_MODE=$(ZFP_ROUNDING_MODE) @@ -127,13 +145,6 @@ ifdef ZFP_ROUNDING_MODE endif endif -# treat subnormals as zero to avoid overflow -ifdef ZFP_WITH_DAZ - ifneq ($(ZFP_WITH_DAZ),0) - FLAGS += -DZFP_WITH_DAZ - endif -endif - # chroma mode for ppm example ifdef PPM_CHROMA PPM_FLAGS += -DPPM_CHROMA=$(PPM_CHROMA) From a7c6047fa29bd4a7da34b4b6892fcfabb4db51a0 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Feb 2022 13:00:14 -0800 Subject: [PATCH 037/277] Attempted fix for issue #150 (cmocka macOS build) --- tests/CMakeLists.txt | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index fc1f72e0e..aa00ef04b 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -9,6 +9,13 @@ endif() # clone cmocka 1.1.0 into /build list(APPEND CMOCKA_ARGS "-DWITH_STATIC_LIB=ON;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") +# fix for zfp issue #150 where CMAKE_BUILD_TYPE is not propagated on macOS +if(MSVC) + set(CMOCKA_BUILD_TYPE "--config ${CMAKE_BUILD_TYPE}") +else() + set(CMOCKA_BUILD_TYPE "") +endif() + include(ExternalProject) ExternalProject_Add( cmocka_cloned @@ -17,7 +24,7 @@ ExternalProject_Add( SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" CMAKE_ARGS ${CMOCKA_ARGS} - BUILD_COMMAND ${CMAKE_COMMAND} --build . --config ${CMAKE_BUILD_TYPE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static + BUILD_COMMAND ${CMAKE_COMMAND} --build . ${CMOCKA_BUILD_TYPE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static INSTALL_COMMAND "" STEP_TARGETS build EXCLUDE_FROM_ALL TRUE From e620e546d5b0f8bc084b0822c429354566e9abaa Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Feb 2022 16:32:20 -0800 Subject: [PATCH 038/277] Undo failed attempt to fix #150 --- tests/CMakeLists.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index aa00ef04b..c0249062a 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -9,7 +9,7 @@ endif() # clone cmocka 1.1.0 into /build list(APPEND CMOCKA_ARGS "-DWITH_STATIC_LIB=ON;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") -# fix for zfp issue #150 where CMAKE_BUILD_TYPE is not propagated on macOS +# WIP: fix for zfp issue #150 where CMAKE_BUILD_TYPE is not propagated on macOS if(MSVC) set(CMOCKA_BUILD_TYPE "--config ${CMAKE_BUILD_TYPE}") else() @@ -24,7 +24,7 @@ ExternalProject_Add( SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" CMAKE_ARGS ${CMOCKA_ARGS} - BUILD_COMMAND ${CMAKE_COMMAND} --build . ${CMOCKA_BUILD_TYPE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static + BUILD_COMMAND ${CMAKE_COMMAND} --build . WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static INSTALL_COMMAND "" STEP_TARGETS build EXCLUDE_FROM_ALL TRUE From 6bfd003aba3012b25d1624abbaf668be38eee6b6 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 9 Feb 2022 11:00:01 -0800 Subject: [PATCH 039/277] Revert attempted fix for #150 --- tests/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index c0249062a..56ebbd6cd 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -24,7 +24,7 @@ ExternalProject_Add( SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" CMAKE_ARGS ${CMOCKA_ARGS} - BUILD_COMMAND ${CMAKE_COMMAND} --build . WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static + BUILD_COMMAND ${CMAKE_COMMAND} --build . --config ${CMAKE_BUILD_TYPE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static INSTALL_COMMAND "" STEP_TARGETS build EXCLUDE_FROM_ALL TRUE From d36f54d9b48db6183718faa04f24e935940b35ba Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 9 Feb 2022 13:23:04 -0800 Subject: [PATCH 040/277] Fixes #150 --- tests/CMakeLists.txt | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 56ebbd6cd..2510e2d5d 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -9,22 +9,14 @@ endif() # clone cmocka 1.1.0 into /build list(APPEND CMOCKA_ARGS "-DWITH_STATIC_LIB=ON;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") -# WIP: fix for zfp issue #150 where CMAKE_BUILD_TYPE is not propagated on macOS -if(MSVC) - set(CMOCKA_BUILD_TYPE "--config ${CMAKE_BUILD_TYPE}") -else() - set(CMOCKA_BUILD_TYPE "") -endif() - include(ExternalProject) ExternalProject_Add( cmocka_cloned - GIT_REPOSITORY https://git.cryptomilk.org/projects/cmocka.git + GIT_REPOSITORY https://gitlab.com/cmocka/cmocka.git GIT_TAG cmocka-1.1.0 SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" - CMAKE_ARGS ${CMOCKA_ARGS} - BUILD_COMMAND ${CMAKE_COMMAND} --build . --config ${CMAKE_BUILD_TYPE} WORKING_DIRECTORY ${CMAKE_BINARY_DIR} cmocka_static + CMAKE_ARGS "${CMOCKA_ARGS}" INSTALL_COMMAND "" STEP_TARGETS build EXCLUDE_FROM_ALL TRUE From f6880a6a661695f91cd39e3cff20f7a8d7c677e1 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 9 Feb 2022 15:00:02 -0800 Subject: [PATCH 041/277] Add #150 to CHANGELOG --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index fdf470a2b..fd7e175da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,7 @@ few of the API changes, other than to cfp, should impact existing code. - #125: OpenMP bit offsets are limited to 32 bits. - #126: `make install` does not install Fortran module. - #127: Reversible mode reports incorrect compressed block size. +- #150: cmocka tests do not build on macOS. - `ZFP_MAX_BITS` is off by one. - `diffusionC`, `iteratorC` are not being built with `gmake`. From ad926b6a857b096ff9b0a5ad7eb7961eb71e9074 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 9 Feb 2022 15:03:10 -0800 Subject: [PATCH 042/277] Add back references to execution section in docs --- docs/source/high-level-api.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 110b8fbbb..223df3e25 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -619,7 +619,7 @@ Execution Policy .. c:function:: zfp_exec_policy zfp_stream_execution(const zfp_stream* stream) - Return current execution policy. + Return current :ref:`execution policy `. ---- @@ -639,9 +639,9 @@ Execution Policy .. c:function:: zfp_bool zfp_stream_set_execution(zfp_stream* stream, zfp_exec_policy policy) - Set execution policy. If different from the previous policy, initialize - the execution parameters to their default values. :code:`zfp_true` is - returned if the execution policy is supported. + Set :ref:`execution policy `. If different from the previous + policy, initialize the execution parameters to their default values. + :code:`zfp_true` is returned if the execution policy is supported. ---- From 19b261d6cbdfba6140b6070b314e561835a68ace Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 11 Feb 2022 11:19:06 -0800 Subject: [PATCH 043/277] Add macros for constructing version number and string --- docs/source/high-level-api.rst | 21 ++++++++++++++++----- include/zfp/version.h | 20 ++++++++++++++------ 2 files changed, 30 insertions(+), 11 deletions(-) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 223df3e25..1500a1f70 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -41,12 +41,24 @@ Macros .. c:macro:: ZFP_VERSION_STRING Macros identifying the |zfp| library version. :c:macro:`ZFP_VERSION` is - a single integer constructed from the previous three macros. - :c:macro:`ZFP_VERSION_STRING` is a string literal. See also + a single integer constructed from the previous three macros + (see :c:macro:`ZFP_MAKE_VERSION`). :c:macro:`ZFP_VERSION_STRING` is a + string literal (see :c:macro:`ZFP_MAKE_VERSION_STRING`). See also :c:data:`zfp_library_version` and :c:data:`zfp_version_string`. ---- +.. c:macro:: ZFP_MAKE_VERSION(major, minor, patch) +.. c:macro:: ZFP_MAKE_VERSION_STRING(major, minor, patch) + + Utility macros for constructing :c:macro:`ZFP_VERSION` and + :c:macro:`ZFP_VERSION_STRING`, respectively. Available as of + |zfp| |64bitrelease|, these macros may be used by applications to test + for a certain |zfp| version number, e.g., + :code:`#if ZFP_VERSION >= ZFP_MAKE_VERSION(0, 5, 6)`. + +---- + .. c:macro:: ZFP_CODEC Macro identifying the version of the compression CODEC. See also @@ -426,9 +438,8 @@ Constants .. c:var:: const char* const zfp_version_string A constant string representing the |zfp| library version and release date. - One can search for this string in executables and libraries that use |zfp| - to determine which version of the library the application was compiled - against. + One can search for this string in executables and libraries that link to + |libzfp| when built as a static library. .. _hl-functions: diff --git a/include/zfp/version.h b/include/zfp/version.h index 87a1d2210..294cc151e 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -5,6 +5,18 @@ #define _zfp_str_(x) # x #define _zfp_str(x) _zfp_str_(x) +/* macro for generating an integer version identifier */ +#define ZFP_MAKE_VERSION(major, minor, patch) \ + (((major) << 8) + \ + ((minor) << 4) + \ + ((patch) << 0)) + +/* macro for generating a version string */ +#define ZFP_MAKE_VERSION_STRING(major, minor, patch) \ + _zfp_str(major) "." \ + _zfp_str(minor) "." \ + _zfp_str(patch) + /* library version information */ #define ZFP_VERSION_MAJOR 0 /* library major version number */ #define ZFP_VERSION_MINOR 5 /* library minor version number */ @@ -16,14 +28,10 @@ /* library version number (see also zfp_library_version) */ #define ZFP_VERSION \ - ((ZFP_VERSION_MAJOR << 8) + \ - (ZFP_VERSION_MINOR << 4) + \ - (ZFP_VERSION_PATCH << 0)) + ZFP_MAKE_VERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) /* library version string (see also zfp_version_string) */ #define ZFP_VERSION_STRING \ - _zfp_str(ZFP_VERSION_MAJOR) "." \ - _zfp_str(ZFP_VERSION_MINOR) "." \ - _zfp_str(ZFP_VERSION_PATCH) + ZFP_MAKE_VERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) #endif From 76d684ded892f621b7dd250cfc6ee322c4c1912d Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 11 Feb 2022 14:15:49 -0800 Subject: [PATCH 044/277] Update CMakeLists to support OpenMP with AppleClang --- tests/src/endtoend/CMakeLists.txt | 3 +-- tests/src/execPolicy/CMakeLists.txt | 7 ++----- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/tests/src/endtoend/CMakeLists.txt b/tests/src/endtoend/CMakeLists.txt index e3849f1ca..5c9aca20f 100644 --- a/tests/src/endtoend/CMakeLists.txt +++ b/tests/src/endtoend/CMakeLists.txt @@ -20,12 +20,11 @@ function(zfp_add_test dims type bits) if(ZFP_WITH_OPENMP) set(omp_test_name testZfpOmp${dims}d${type}) add_executable(${omp_test_name} ${omp_test_name}.c) - target_compile_options(${omp_test_name} PRIVATE ${OpenMP_C_FLAGS}) target_compile_definitions(${omp_test_name} PRIVATE ${zfp_private_defs}) target_link_libraries(${omp_test_name} cmocka zfp zfpHashLib genSmoothRandNumsLib stridedOperationsLib zfpChecksumsLib zfpTimerLib zfpCompressionParamsLib - ${OpenMP_C_LIBRARIES}) + OpenMP::OpenMP_C) if(HAVE_LIBM_MATH) target_link_libraries(${omp_test_name} m) endif() diff --git a/tests/src/execPolicy/CMakeLists.txt b/tests/src/execPolicy/CMakeLists.txt index f37e16acc..5e49a46c3 100644 --- a/tests/src/execPolicy/CMakeLists.txt +++ b/tests/src/execPolicy/CMakeLists.txt @@ -2,16 +2,13 @@ add_executable(testOmp testOmp.c) target_link_libraries(testOmp cmocka zfp) add_test(NAME testOmp COMMAND testOmp) if(ZFP_WITH_OPENMP) - target_compile_options(testOmp PRIVATE ${OpenMP_C_FLAGS}) - target_link_libraries(testOmp ${OpenMP_C_LIBRARIES}) + target_link_libraries(testOmp OpenMP::OpenMP_C) set_property(TEST testOmp PROPERTY RUN_SERIAL TRUE) endif() if(ZFP_WITH_OPENMP) add_executable(testOmpInternal testOmpInternal.c) - target_compile_options(testOmpInternal PRIVATE ${OpenMP_C_FLAGS}) - target_link_libraries(testOmpInternal - cmocka zfp ${OpenMP_C_FLAGS} ${OpenMP_C_LIBRARIES}) + target_link_libraries(testOmpInternal cmocka zfp OpenMP::OpenMP_C) add_test(NAME testOmpInternal COMMAND testOmpInternal) endif() From baaa782acc0fb7c1f2aad2c0ad6e8b6292df341d Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 11 Feb 2022 14:54:40 -0800 Subject: [PATCH 045/277] Remove debugging code from diffusion example --- examples/diffusion.cpp | 33 --------------------------------- 1 file changed, 33 deletions(-) diff --git a/examples/diffusion.cpp b/examples/diffusion.cpp index 7b8bc150f..cd3fb362e 100644 --- a/examples/diffusion.cpp +++ b/examples/diffusion.cpp @@ -91,7 +91,6 @@ template <> inline void time_step_parallel(zfp::array2d& u, zfp::array2d& du, const Constants& c) { -#if 0 // flush shared cache to ensure cache consistency across threads u.flush_cache(); // zero-initialize du @@ -104,7 +103,6 @@ time_step_parallel(zfp::array2d& u, zfp::array2d& du, const Constants& c) // create read-write private view into rectangular subset of du zfp::array2d::private_view mydu(&du); mydu.partition(omp_get_thread_num(), omp_get_num_threads()); -//fprintf(stderr, "thread=%d (%zu, %zu) (%zu, %zu)\n", omp_get_thread_num(), mydu.global_x(0), mydu.global_y(0), mydu.size_x(), mydu.size_y()); // process rectangular region owned by this thread for (size_t j = 0; j < mydu.size_y(); j++) { size_t y = mydu.global_y(j); @@ -121,37 +119,6 @@ time_step_parallel(zfp::array2d& u, zfp::array2d& du, const Constants& c) // take forward Euler step in serial for (size_t i = 0; i < u.size(); i++) u[i] += du[i]; -#else - // flush shared cache to ensure cache consistency across threads - u.flush_cache(); - // zero-initialize du - du.set(0); - // compute du/dt in parallel - uint omp_num_threads = 8; - for (uint omp_thread_num = 0; omp_thread_num < omp_num_threads; omp_thread_num++) - { - // create read-only private view of entire array u - zfp::array2d::private_const_view myu(&u); - // create read-write private view into rectangular subset of du - zfp::array2d::private_view mydu(&du); - mydu.partition(omp_thread_num, omp_num_threads); - // process rectangular region owned by this thread - for (size_t j = 0; j < mydu.size_y(); j++) { - size_t y = mydu.global_y(j); - if (1 <= y && y <= c.ny - 2) - for (size_t i = 0; i < mydu.size_x(); i++) { - size_t x = mydu.global_x(i); - if (1 <= x && x <= c.nx - 2) - mydu(i, j) = c.dt * c.k * laplacian(myu, x, y, c); - } - } - // compress all private cached blocks to shared storage - mydu.flush_cache(); - } - // take forward Euler step in serial - for (size_t i = 0; i < u.size(); i++) - u[i] += du[i]; -#endif } #else // dummy template instantiation when OpenMP support is not available From f49a7fe5d8a137e1b36b044797a5e4234c270e3b Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 11 Feb 2022 15:09:46 -0800 Subject: [PATCH 046/277] Silence compiler warnings in tests --- tests/src/misc/testZfpPromote.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tests/src/misc/testZfpPromote.c b/tests/src/misc/testZfpPromote.c index 567580e8d..6416abee1 100644 --- a/tests/src/misc/testZfpPromote.c +++ b/tests/src/misc/testZfpPromote.c @@ -24,9 +24,9 @@ given_int8_when_promoteToInt32_expect_demoteToInt8Matches(void **state) assert_non_null(oblock8); assert_non_null(block32); - int8 i; + uint i; for (i = 0; i < sz; i++) - iblock8[i] = i; + iblock8[i] = (int8)i; zfp_promote_int8_to_int32(block32, iblock8, dims); zfp_demote_int32_to_int8(oblock8, block32, dims); @@ -48,9 +48,9 @@ given_uint8_when_promoteToInt32_expect_demoteToUInt8Matches(void **state) assert_non_null(oblock8); assert_non_null(block32); - uint8 i; + uint i; for (i = 0; i < sz; i++) - iblock8[i] = i; + iblock8[i] = (uint8)i; zfp_promote_uint8_to_int32(block32, iblock8, dims); zfp_demote_int32_to_uint8(oblock8, block32, dims); @@ -72,9 +72,9 @@ given_int16_when_promoteToInt32_expect_demoteToInt16Matches(void **state) assert_non_null(oblock16); assert_non_null(block32); - int16 i; + uint i; for (i = 0; i < sz; i++) - iblock16[i] = i; + iblock16[i] = (int16)i; zfp_promote_int16_to_int32(block32, iblock16, dims); zfp_demote_int32_to_int16(oblock16, block32, dims); @@ -96,9 +96,9 @@ given_uint16_when_promoteToInt32_expect_demoteToUInt16Matches(void **state) assert_non_null(oblock16); assert_non_null(block32); - uint16 i; + uint i; for (i = 0; i < sz; i++) - iblock16[i] = i; + iblock16[i] = (uint16)i; zfp_promote_uint16_to_int32(block32, iblock16, dims); zfp_demote_int32_to_uint16(oblock16, block32, dims); From 1d6fbe597b1d86e78b51ffd8c13e2068a0c74b0f Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 11 Feb 2022 15:13:03 -0800 Subject: [PATCH 047/277] Silence compiler warnings in tests --- tests/array/encode/testTemplatedEncodeBase.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/array/encode/testTemplatedEncodeBase.cpp b/tests/array/encode/testTemplatedEncodeBase.cpp index e4841d6c2..08ecd24d0 100644 --- a/tests/array/encode/testTemplatedEncodeBase.cpp +++ b/tests/array/encode/testTemplatedEncodeBase.cpp @@ -192,7 +192,7 @@ bool streamsEqual(zfp_stream** stream1, zfp_stream** stream2) char* data2 = (char*)stream_data(s2); zfp_stream_flush(*stream2); - for (int i = 0; i < sz1; i++) + for (size_t i = 0; i < sz1; i++) if (data1[i] != data2[i]) return false; return true; From 0f06782e4bac6aebb5a66eb6dd4b4a32234b2109 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 11 Feb 2022 15:51:30 -0800 Subject: [PATCH 048/277] Rename word -> stream_word --- docs/source/bit-stream.rst | 21 +++--- src/inline/bitstream.c | 50 ++++++------- tests/src/inline/testBitstream.c | 82 +++++++++++----------- tests/src/inline/testBitstreamSmallWsize.c | 6 +- tests/src/inline/testBitstreamStrided.c | 8 +-- 5 files changed, 86 insertions(+), 81 deletions(-) diff --git a/docs/source/bit-stream.rst b/docs/source/bit-stream.rst index 19c5054a2..9dd2187e2 100644 --- a/docs/source/bit-stream.rst +++ b/docs/source/bit-stream.rst @@ -66,7 +66,7 @@ section. Types ----- -.. c:type:: word +.. c:type:: stream_word Bits are buffered and read/written in units of words. By default, the bit stream word type is 64 bits, but may be set to 8, 16, or 32 bits @@ -75,6 +75,11 @@ Types tend to give higher throughput, while 8-bit words are needed to ensure endian independence (see FAQ :ref:`#11 `). +.. note:: + To avoid potential name clashes, this type was renamed in + |zfp| |64bitrelease| from the shorter and more ambiguous type name + :code:`word`. + ---- .. c:type:: bitstream @@ -85,13 +90,13 @@ Types :: struct bitstream { - uint bits; // number of buffered bits (0 <= bits < word size) - word buffer; // buffer for incoming/outgoing bits (buffer < 2^bits) - word* ptr; // pointer to next word to be read/written - word* begin; // beginning of stream - word* end; // end of stream (currently unused) - size_t mask; // one less the block size in number of words (if BIT_STREAM_STRIDED) - ptrdiff_t delta; // number of words between consecutive blocks (if BIT_STREAM_STRIDED) + uint bits; // number of buffered bits (0 <= bits < word size) + stream_word buffer; // buffer for incoming/outgoing bits (buffer < 2^bits) + stream_word* ptr; // pointer to next word to be read/written + stream_word* begin; // beginning of stream + stream_word* end; // end of stream (currently unused) + size_t mask; // one less the block size in number of words (if BIT_STREAM_STRIDED) + ptrdiff_t delta; // number of words between consecutive blocks (if BIT_STREAM_STRIDED) }; .. _bs-data: diff --git a/src/inline/bitstream.c b/src/inline/bitstream.c index 33fceb817..3293ac343 100644 --- a/src/inline/bitstream.c +++ b/src/inline/bitstream.c @@ -113,35 +113,35 @@ The following assumptions and restrictions apply: /* bit stream word/buffer type; granularity of stream I/O operations */ #ifdef BIT_STREAM_WORD_TYPE /* may be 8-, 16-, 32-, or 64-bit unsigned integer type */ - typedef BIT_STREAM_WORD_TYPE word; + typedef BIT_STREAM_WORD_TYPE stream_word; #else /* use maximum word size by default for highest speed */ - typedef uint64 word; + typedef uint64 stream_word; #endif /* number of bits in a buffered word */ -#define wsize ((uint)(CHAR_BIT * sizeof(word))) +#define wsize ((uint)(CHAR_BIT * sizeof(stream_word))) /* bit stream structure (opaque to caller) */ struct bitstream { - uint bits; /* number of buffered bits (0 <= bits < wsize) */ - word buffer; /* buffer for incoming/outgoing bits (buffer < 2^bits) */ - word* ptr; /* pointer to next word to be read/written */ - word* begin; /* beginning of stream */ - word* end; /* end of stream (currently unused) */ + uint bits; /* number of buffered bits (0 <= bits < wsize) */ + stream_word buffer; /* buffer for incoming/outgoing bits (buffer < 2^bits) */ + stream_word* ptr; /* pointer to next word to be read/written */ + stream_word* begin; /* beginning of stream */ + stream_word* end; /* end of stream (currently unused) */ #ifdef BIT_STREAM_STRIDED - size_t mask; /* one less the block size in number of words */ - ptrdiff_t delta; /* number of words between consecutive blocks */ + size_t mask; /* one less the block size in number of words */ + ptrdiff_t delta; /* number of words between consecutive blocks */ #endif }; /* private functions ------------------------------------------------------- */ /* read a single word from memory */ -static word +static stream_word stream_read_word(bitstream* s) { - word w = *s->ptr++; + stream_word w = *s->ptr++; #ifdef BIT_STREAM_STRIDED if (!((s->ptr - s->begin) & s->mask)) s->ptr += s->delta; @@ -151,7 +151,7 @@ stream_read_word(bitstream* s) /* write a single word to memory */ static void -stream_write_word(bitstream* s, word value) +stream_write_word(bitstream* s, stream_word value) { *s->ptr++ = value; #ifdef BIT_STREAM_STRIDED @@ -180,14 +180,14 @@ stream_data(const bitstream* s) inline_ size_t stream_size(const bitstream* s) { - return sizeof(word) * (size_t)(s->ptr - s->begin); + return sizeof(stream_word) * (size_t)(s->ptr - s->begin); } /* byte capacity of stream */ inline_ size_t stream_capacity(const bitstream* s) { - return sizeof(word) * (size_t)(s->end - s->begin); + return sizeof(stream_word) * (size_t)(s->end - s->begin); } /* number of words per block */ @@ -233,7 +233,7 @@ stream_read_bit(bitstream* s) inline_ uint stream_write_bit(bitstream* s, uint bit) { - s->buffer += (word)bit << s->bits; + s->buffer += (stream_word)bit << s->bits; if (++s->bits == wsize) { stream_write_word(s, s->buffer); s->buffer = 0; @@ -282,7 +282,7 @@ inline_ uint64 stream_write_bits(bitstream* s, uint64 value, uint n) { /* append bit string to buffer */ - s->buffer += (word)(value << s->bits); + s->buffer += (stream_word)(value << s->bits); s->bits += n; /* is buffer full? */ if (s->bits >= wsize) { @@ -296,11 +296,11 @@ stream_write_bits(bitstream* s, uint64 value, uint n) /* assert: 0 <= s->bits <= n */ stream_write_word(s, s->buffer); /* assert: 0 <= n - s->bits < 64 */ - s->buffer = (word)(value >> (n - s->bits)); + s->buffer = (stream_word)(value >> (n - s->bits)); } while (sizeof(s->buffer) < sizeof(value) && s->bits >= wsize); } /* assert: 0 <= s->bits < wsize */ - s->buffer &= ((word)1 << s->bits) - 1; + s->buffer &= ((stream_word)1 << s->bits) - 1; /* assert: 0 <= n < 64 */ return value >> n; } @@ -351,8 +351,8 @@ stream_wseek(bitstream* s, size_t offset) uint n = offset % wsize; s->ptr = s->begin + offset / wsize; if (n) { - word buffer = *s->ptr; - buffer &= ((word)1 << n) - 1; + stream_word buffer = *s->ptr; + buffer &= ((stream_word)1 << n) - 1; s->buffer = buffer; s->bits = n; } @@ -404,12 +404,12 @@ inline_ void stream_copy(bitstream* dst, bitstream* src, size_t n) { while (n > wsize) { - word w = (word)stream_read_bits(src, wsize); + stream_word w = (stream_word)stream_read_bits(src, wsize); stream_write_bits(dst, w, wsize); n -= wsize; } if (n) { - word w = (word)stream_read_bits(src, (uint)n); + stream_word w = (stream_word)stream_read_bits(src, (uint)n); stream_write_bits(dst, w, (uint)n); } } @@ -434,8 +434,8 @@ stream_open(void* buffer, size_t bytes) { bitstream* s = (bitstream*)malloc(sizeof(bitstream)); if (s) { - s->begin = (word*)buffer; - s->end = s->begin + bytes / sizeof(word); + s->begin = (stream_word*)buffer; + s->end = s->begin + bytes / sizeof(stream_word); #ifdef BIT_STREAM_STRIDED stream_set_stride(s, 0, 0); #endif diff --git a/tests/src/inline/testBitstream.c b/tests/src/inline/testBitstream.c index 96ac9c551..948a77259 100644 --- a/tests/src/inline/testBitstream.c +++ b/tests/src/inline/testBitstream.c @@ -8,7 +8,7 @@ #define STREAM_WORD_CAPACITY 3 -#define WORD_MASK ((word)(-1)) +#define WORD_MASK ((stream_word)(-1)) #define WORD1 WORD_MASK #define WORD2 (0x5555555555555555 & WORD_MASK) @@ -23,10 +23,10 @@ setup(void **state) struct setupVars *s = malloc(sizeof(struct setupVars)); assert_non_null(s); - s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(word)); + s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(stream_word)); assert_non_null(s->buffer); - s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(word)); + s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(stream_word)); assert_non_null(s->b); *state = s; @@ -53,11 +53,11 @@ when_StreamCopy_expect_BitsCopiedToDestBitstream(void **state) const uint COPY_BITS = wsize + 4; const uint NUM_WORD2_BITS_WRITTEN_TO_WORD = DST_OFFSET + (wsize - SRC_OFFSET); - const word EXPECTED_WRITTEN_WORD = ((WORD1 >> SRC_OFFSET) << DST_OFFSET) - + (WORD2 << NUM_WORD2_BITS_WRITTEN_TO_WORD); + const stream_word EXPECTED_WRITTEN_WORD = ((WORD1 >> SRC_OFFSET) << DST_OFFSET) + + (WORD2 << NUM_WORD2_BITS_WRITTEN_TO_WORD); const uint EXPECTED_BITS = (DST_OFFSET + COPY_BITS) % wsize; - const word EXPECTED_BUFFER = (WORD2 >> (NUM_WORD2_BITS_WRITTEN_TO_WORD)) - & ((1u << EXPECTED_BITS) - 1); + const stream_word EXPECTED_BUFFER = (WORD2 >> (NUM_WORD2_BITS_WRITTEN_TO_WORD)) + & ((1u << EXPECTED_BITS) - 1); bitstream* src = ((struct setupVars *)*state)->b; stream_write_word(src, WORD1); @@ -65,8 +65,8 @@ when_StreamCopy_expect_BitsCopiedToDestBitstream(void **state) stream_flush(src); stream_rseek(src, SRC_OFFSET); - void* buffer = calloc(STREAM_WORD_CAPACITY, sizeof(word)); - bitstream* dst = stream_open(buffer, STREAM_WORD_CAPACITY * sizeof(word)); + void* buffer = calloc(STREAM_WORD_CAPACITY, sizeof(stream_word)); + bitstream* dst = stream_open(buffer, STREAM_WORD_CAPACITY * sizeof(stream_word)); stream_wseek(dst, DST_OFFSET); stream_copy(dst, src, COPY_BITS); @@ -91,7 +91,7 @@ when_Flush_expect_PaddedWordWrittenToStream(void **state) stream_rewind(s); stream_write_bits(s, WORD2, PREV_BUFFER_BIT_COUNT); - word *prevPtr = s->ptr; + stream_word *prevPtr = s->ptr; size_t padCount = stream_flush(s); @@ -105,9 +105,9 @@ static void given_EmptyBuffer_when_Flush_expect_NOP(void **state) { bitstream* s = ((struct setupVars *)*state)->b; - word *prevPtr = s->ptr; + stream_word *prevPtr = s->ptr; uint prevBits = s->bits; - word prevBuffer = s->buffer; + stream_word prevBuffer = s->buffer; size_t padCount = stream_flush(s); @@ -128,7 +128,7 @@ when_Align_expect_BufferEmptyBitsZero(void **state) stream_rewind(s); stream_read_bits(s, READ_BIT_COUNT); - word *prevPtr = s->ptr; + stream_word *prevPtr = s->ptr; stream_align(s); @@ -144,7 +144,7 @@ when_SkipPastBufferEnd_expect_NewMaskedWordInBuffer(void **state) const uint SKIP_COUNT = wsize + 5; const uint TOTAL_OFFSET = READ_BIT_COUNT + SKIP_COUNT; const uint EXPECTED_BITS = wsize - (TOTAL_OFFSET % wsize); - const word EXPECTED_BUFFER = WORD2 >> (TOTAL_OFFSET % wsize); + const stream_word EXPECTED_BUFFER = WORD2 >> (TOTAL_OFFSET % wsize); bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); @@ -167,14 +167,14 @@ when_SkipWithinBuffer_expect_MaskedBuffer(void **state) const uint SKIP_COUNT = 5; const uint TOTAL_OFFSET = READ_BIT_COUNT + SKIP_COUNT; const uint EXPECTED_BITS = wsize - (TOTAL_OFFSET % wsize); - const word EXPECTED_BUFFER = WORD1 >> (TOTAL_OFFSET % wsize); + const stream_word EXPECTED_BUFFER = WORD1 >> (TOTAL_OFFSET % wsize); bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); stream_rewind(s); stream_read_bits(s, READ_BIT_COUNT); - word *prevPtr = s->ptr; + stream_word *prevPtr = s->ptr; stream_skip(s, SKIP_COUNT); @@ -193,9 +193,9 @@ when_SkipZeroBits_expect_NOP(void **state) stream_rewind(s); stream_read_bits(s, 2); - word* prevPtr = s->ptr; - word prevBits = s->bits; - word prevBuffer = s->buffer; + stream_word* prevPtr = s->ptr; + stream_word prevBits = s->bits; + stream_word prevBuffer = s->buffer; stream_skip(s, 0); @@ -209,7 +209,7 @@ when_RseekToNonMultipleOfWsize_expect_MaskedWordLoadedToBuffer(void **state) { const uint BIT_OFFSET = wsize + 5; const uint EXPECTED_BITS = wsize - (BIT_OFFSET % wsize); - const word EXPECTED_BUFFER = WORD2 >> (BIT_OFFSET % wsize); + const stream_word EXPECTED_BUFFER = WORD2 >> (BIT_OFFSET % wsize); bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); @@ -240,7 +240,7 @@ static void when_WseekToNonMultipleOfWsize_expect_MaskedWordLoadedToBuffer(void **state) { const uint BIT_OFFSET = wsize + 5; - const word MASK = 0x1f; + const stream_word MASK = 0x1f; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); @@ -305,8 +305,8 @@ when_ReadBitsSpreadsAcrossTwoWords_expect_BitsCombinedFromBothWords(void **state const uint NUM_OVERFLOWED_BITS = READ_BIT_COUNT - PARTIAL_WORD_BIT_COUNT; const uint EXPECTED_BUFFER_BIT_COUNT = wsize - NUM_OVERFLOWED_BITS; - const word PARTIAL_WORD1 = WORD1 & 0xffff; - const word PARTIAL_WORD2 = WORD2 & 0x1fffffffffff << PARTIAL_WORD_BIT_COUNT; + const stream_word PARTIAL_WORD1 = WORD1 & 0xffff; + const stream_word PARTIAL_WORD2 = WORD2 & 0x1fffffffffff << PARTIAL_WORD_BIT_COUNT; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, PARTIAL_WORD1, wsize); @@ -347,7 +347,7 @@ static void when_ReadBits_expect_BitsReadInOrderLSB(void **state) { const uint BITS_TO_READ = 2; - const word MASK = 0x3; + const stream_word MASK = 0x3; bitstream* s = ((struct setupVars *)*state)->b; s->buffer = WORD2; @@ -383,9 +383,9 @@ when_WriteBitsOverflowsBuffer_expect_OverflowWrittenToNewBuffer(void **state) const uint NUM_BITS_TO_WRITE = wsize - 1; const uint OVERFLOW_BIT_COUNT = NUM_BITS_TO_WRITE - (wsize - EXISTING_BIT_COUNT); // 0x1101 0101 0101 ... 0101 allows stream_write_bit() to return non-zero - const word WORD_TO_WRITE = WORD2 + 0x8000000000000000; - const word OVERFLOWED_BITS = WORD_TO_WRITE >> (wsize - EXISTING_BIT_COUNT); - const word EXPECTED_BUFFER_RESULT = OVERFLOWED_BITS & 0xf; + const stream_word WORD_TO_WRITE = WORD2 + 0x8000000000000000; + const stream_word OVERFLOWED_BITS = WORD_TO_WRITE >> (wsize - EXISTING_BIT_COUNT); + const stream_word EXPECTED_BUFFER_RESULT = OVERFLOWED_BITS & 0xf; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, EXISTING_BIT_COUNT); @@ -402,14 +402,14 @@ when_WriteBitsFillsBufferExactly_expect_WordWrittenToStream(void **state) { const uint EXISTING_BIT_COUNT = 5; const uint NUM_BITS_TO_WRITE = wsize - EXISTING_BIT_COUNT; - const word COMPLETING_WORD = WORD2 & 0x07ffffffffffffff; + const stream_word COMPLETING_WORD = WORD2 & 0x07ffffffffffffff; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, EXISTING_BIT_COUNT); uint64 remainingBits = stream_write_bits(s, COMPLETING_WORD, NUM_BITS_TO_WRITE); stream_rewind(s); - word readWord = stream_read_word(s); + stream_word readWord = stream_read_word(s); assert_int_equal(readWord, 0x1f + 0xaaaaaaaaaaaaaaa0); assert_int_equal(remainingBits, 0); @@ -465,7 +465,7 @@ given_BitstreamWithBitInBuffer_when_ReadBit_expect_OneBitReadFromLSB(void **stat stream_write_bit(s, 1); uint prevBits = s->bits; - word prevBuffer = s->buffer; + stream_word prevBuffer = s->buffer; assert_int_equal(stream_read_bit(s), 1); assert_int_equal(s->bits, prevBits - 1); @@ -482,8 +482,8 @@ given_BitstreamBufferOneBitFromFull_when_WriteBit_expect_BitWrittenToBufferWritt stream_write_bit(s, 1); - assert_int_equal(stream_size(s), sizeof(word)); - assert_int_equal(*s->begin, (word)1 << PLACE); + assert_int_equal(stream_size(s), sizeof(stream_word)); + assert_int_equal(*s->begin, (stream_word)1 << PLACE); assert_int_equal(s->buffer, 0); } @@ -498,7 +498,7 @@ when_WriteBit_expect_BitWrittenToBufferFromLSB(void **state) stream_write_bit(s, 1); assert_int_equal(s->bits, PLACE + 1); - assert_int_equal(s->buffer, (word)1 << PLACE); + assert_int_equal(s->buffer, (stream_word)1 << PLACE); } static void @@ -506,7 +506,7 @@ given_StartedBuffer_when_StreamPadOverflowsBuffer_expect_ProperWordsWritten(void { const uint NUM_WORDS = 2; const uint EXISTING_BIT_COUNT = 12; - const word EXISTING_BUFFER = 0xfff; + const stream_word EXISTING_BUFFER = 0xfff; const uint PAD_AMOUNT = NUM_WORDS * wsize - EXISTING_BIT_COUNT; bitstream* s = ((struct setupVars *)*state)->b; @@ -520,7 +520,7 @@ given_StartedBuffer_when_StreamPadOverflowsBuffer_expect_ProperWordsWritten(void stream_pad(s, PAD_AMOUNT); - assert_int_equal(stream_size(s), prevStreamSize + NUM_WORDS * sizeof(word)); + assert_int_equal(stream_size(s), prevStreamSize + NUM_WORDS * sizeof(stream_word)); stream_rewind(s); assert_int_equal(stream_read_word(s), EXISTING_BUFFER); assert_int_equal(stream_read_word(s), 0); @@ -530,7 +530,7 @@ static void given_StartedBuffer_when_StreamPad_expect_PaddedWordWritten(void **state) { const uint EXISTING_BIT_COUNT = 12; - const word EXISTING_BUFFER = 0xfff; + const stream_word EXISTING_BUFFER = 0xfff; bitstream* s = ((struct setupVars *)*state)->b; s->buffer = EXISTING_BUFFER; @@ -539,7 +539,7 @@ given_StartedBuffer_when_StreamPad_expect_PaddedWordWritten(void **state) stream_pad(s, wsize - EXISTING_BIT_COUNT); - assert_int_equal(stream_size(s), prevStreamSize + sizeof(word)); + assert_int_equal(stream_size(s), prevStreamSize + sizeof(stream_word)); stream_rewind(s); assert_int_equal(stream_read_word(s), EXISTING_BUFFER); } @@ -586,7 +586,7 @@ when_WriteTwoWords_expect_WordsWrittenToStreamConsecutively(void **state) stream_write_word(s, WORD1); stream_write_word(s, WORD2); - assert_int_equal(stream_size(s), sizeof(word) * 2); + assert_int_equal(stream_size(s), sizeof(stream_word) * 2); assert_int_equal(*s->begin, WORD1); assert_int_equal(*(s->begin + 1), WORD2); } @@ -599,7 +599,7 @@ given_RewoundBitstream_when_WriteWord_expect_WordWrittenAtStreamBegin(void **sta stream_write_word(s, WORD1); - assert_int_equal(stream_size(s), prevStreamSize + sizeof(word)); + assert_int_equal(stream_size(s), prevStreamSize + sizeof(stream_word)); assert_int_equal(*s->begin, WORD1); } @@ -608,12 +608,12 @@ when_BitstreamOpened_expect_ProperLengthAndBoundaries(void **state) { const int NUM_WORDS = 4; - size_t bufferLenBytes = sizeof(word) * NUM_WORDS; + size_t bufferLenBytes = sizeof(stream_word) * NUM_WORDS; void* buffer = malloc(bufferLenBytes); bitstream* s = stream_open(buffer, bufferLenBytes); void* streamBegin = stream_data(s); - void* computedStreamEnd = (word*)streamBegin + NUM_WORDS; + void* computedStreamEnd = (stream_word*)streamBegin + NUM_WORDS; assert_ptr_equal(streamBegin, buffer); assert_ptr_equal(s->end, computedStreamEnd); diff --git a/tests/src/inline/testBitstreamSmallWsize.c b/tests/src/inline/testBitstreamSmallWsize.c index 3f0c966dd..4bed088e3 100644 --- a/tests/src/inline/testBitstreamSmallWsize.c +++ b/tests/src/inline/testBitstreamSmallWsize.c @@ -21,10 +21,10 @@ setup(void **state) struct setupVars *s = malloc(sizeof(struct setupVars)); assert_non_null(s); - s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(word)); + s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(stream_word)); assert_non_null(s->buffer); - s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(word)); + s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(stream_word)); assert_non_null(s->b); *state = s; @@ -72,7 +72,7 @@ when_ReadBitsSpreadsAcrossMultipleWords_expect_BitsCombinedFromMultipleWords(voi + (WRITE_BITS2 << PARTIAL_WORD_BIT_COUNT) + (WRITE_BITS3 << (wsize + PARTIAL_WORD_BIT_COUNT)) + ((WRITE_BITS4 & 0xff) << (2*wsize + PARTIAL_WORD_BIT_COUNT))); - assert_int_equal(s->buffer, (word) (WRITE_BITS4 >> (NUM_OVERFLOWED_BITS % wsize))); + assert_int_equal(s->buffer, (stream_word) (WRITE_BITS4 >> (NUM_OVERFLOWED_BITS % wsize))); } // overflow refers to what will land in the buffer diff --git a/tests/src/inline/testBitstreamStrided.c b/tests/src/inline/testBitstreamStrided.c index f7349d67d..649211b6b 100644 --- a/tests/src/inline/testBitstreamStrided.c +++ b/tests/src/inline/testBitstreamStrided.c @@ -26,10 +26,10 @@ setup(void **state) struct setupVars *s = malloc(sizeof(struct setupVars)); assert_non_null(s); - s->buffer = calloc(STREAM_STRIDED_LEN, sizeof(word)); + s->buffer = calloc(STREAM_STRIDED_LEN, sizeof(stream_word)); assert_non_null(s->buffer); - s->b = stream_open(s->buffer, STREAM_STRIDED_LEN * sizeof(word)); + s->b = stream_open(s->buffer, STREAM_STRIDED_LEN * sizeof(stream_word)); assert_non_null(s->b); assert_true(stream_set_stride(s->b, BLOCK_SIZE, DELTA)); @@ -54,7 +54,7 @@ static void given_Strided_when_ReadWordCompletesBlock_expect_PtrAdvancedByStrideLen(void **state) { bitstream* s = ((struct setupVars *)*state)->b; - word* prevPtr = s->ptr; + stream_word* prevPtr = s->ptr; int i; for (i = 0; i < BLOCK_SIZE - 1; i++) { @@ -71,7 +71,7 @@ static void given_Strided_when_WriteWordCompletesBlock_expect_PtrAdvancedByStrideLen(void **state) { bitstream* s = ((struct setupVars *)*state)->b; - word* prevPtr = s->ptr; + stream_word* prevPtr = s->ptr; int i; for (i = 0; i < BLOCK_SIZE - 1; i++) { From 15d77178c72232b638610b344bb1b99a0704b446 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Mon, 7 Mar 2022 15:47:05 -0800 Subject: [PATCH 049/277] Move memory management to internal namespace --- array/zfp/cache.h | 12 ++++++------ array/zfp/memory.h | 22 ++++++++++++---------- array/zfp/store.h | 10 +++++----- array/zfpcodec.h | 2 +- array/zfpindex.h | 12 ++++++------ 5 files changed, 30 insertions(+), 28 deletions(-) diff --git a/array/zfp/cache.h b/array/zfp/cache.h index f1e415311..55b368b13 100644 --- a/array/zfp/cache.h +++ b/array/zfp/cache.h @@ -107,8 +107,8 @@ class Cache { // destructor ~Cache() { - zfp::deallocate_aligned(tag); - zfp::deallocate_aligned(line); + zfp::internal::deallocate_aligned(tag); + zfp::internal::deallocate_aligned(line); #ifdef ZFP_WITH_CACHE_PROFILE std::cerr << "cache R1=" << hit[0][0] << " R2=" << hit[1][0] << " RM=" << miss[0] << " RB=" << back[0] << " W1=" << hit[0][1] << " W2=" << hit[1][1] << " WM=" << miss[1] << " WB=" << back[1] << std::endl; @@ -142,8 +142,8 @@ class Cache { { // compute smallest value of mask such that mask + 1 = 2^k >= minsize for (mask = minsize ? minsize - 1 : 1; mask & (mask + 1); mask |= mask + 1); - zfp::reallocate_aligned(tag, size() * sizeof(Tag), ZFP_MEMORY_ALIGNMENT); - zfp::reallocate_aligned(line, size() * sizeof(Line), ZFP_MEMORY_ALIGNMENT); + zfp::internal::reallocate_aligned(tag, size() * sizeof(Tag), ZFP_MEMORY_ALIGNMENT); + zfp::internal::reallocate_aligned(line, size() * sizeof(Line), ZFP_MEMORY_ALIGNMENT); clear(); } @@ -230,8 +230,8 @@ class Cache { void deep_copy(const Cache& c) { mask = c.mask; - zfp::clone_aligned(tag, c.tag, size(), ZFP_MEMORY_ALIGNMENT); - zfp::clone_aligned(line, c.line, size(), ZFP_MEMORY_ALIGNMENT); + zfp::internal::clone_aligned(tag, c.tag, size(), ZFP_MEMORY_ALIGNMENT); + zfp::internal::clone_aligned(line, c.line, size(), ZFP_MEMORY_ALIGNMENT); #ifdef ZFP_WITH_CACHE_PROFILE hit[0][0] = c.hit[0][0]; hit[0][1] = c.hit[0][1]; diff --git a/array/zfp/memory.h b/array/zfp/memory.h index d3c5036af..d9067adf3 100644 --- a/array/zfp/memory.h +++ b/array/zfp/memory.h @@ -26,6 +26,7 @@ extern "C" { #define unused_(x) ((void)(x)) namespace zfp { +namespace internal { // allocate size bytes inline void* @@ -108,8 +109,8 @@ reallocate(T*& ptr, size_t size, bool preserve = false) if (preserve) ptr = static_cast(std::realloc(ptr, size)); else { - zfp::deallocate(ptr); - ptr = static_cast(zfp::allocate(size)); + zfp::internal::deallocate(ptr); + ptr = static_cast(zfp::internal::allocate(size)); } } @@ -130,15 +131,15 @@ reallocate_aligned(void*& ptr, size_t new_size, size_t alignment, size_t old_siz { if (old_size) { // reallocate while preserving contents - void* dst = zfp::allocate_aligned(new_size, alignment); + void* dst = zfp::internal::allocate_aligned(new_size, alignment); std::memcpy(dst, ptr, std::min(old_size, new_size)); - zfp::deallocate_aligned(ptr); + zfp::internal::deallocate_aligned(ptr); ptr = dst; } else { // reallocate without preserving contents - zfp::deallocate_aligned(ptr); - ptr = zfp::allocate_aligned(new_size, alignment); + zfp::internal::deallocate_aligned(ptr); + ptr = zfp::internal::allocate_aligned(new_size, alignment); } } @@ -147,9 +148,9 @@ template inline void clone(T*& dst, const T* src, size_t count) { - zfp::deallocate(dst); + zfp::internal::deallocate(dst); if (src) { - dst = static_cast(zfp::allocate(count * sizeof(T))); + dst = static_cast(zfp::internal::allocate(count * sizeof(T))); std::copy(src, src + count, dst); } else @@ -173,9 +174,9 @@ template <> inline void clone_aligned(void*& dst, const void* src, size_t size, size_t alignment) { - zfp::deallocate_aligned(dst); + zfp::internal::deallocate_aligned(dst); if (src) { - dst = zfp::allocate_aligned(size, alignment); + dst = zfp::internal::allocate_aligned(size, alignment); std::memcpy(dst, src, size); } else @@ -191,6 +192,7 @@ round_up(size_t size, size_t unit) return size; } +} } #undef unused_ diff --git a/array/zfp/store.h b/array/zfp/store.h index 554638051..64a349ac5 100644 --- a/array/zfp/store.h +++ b/array/zfp/store.h @@ -114,10 +114,10 @@ class BlockStore { // shrink buffer to match size of compressed data void compact() { - size_t size = zfp::round_up(index.range(), codec.alignment() * CHAR_BIT) / CHAR_BIT; + size_t size = zfp::internal::round_up(index.range(), codec.alignment() * CHAR_BIT) / CHAR_BIT; if (bytes > size) { codec.close(); - zfp::reallocate_aligned(data, size, ZFP_MEMORY_ALIGNMENT, bytes); + zfp::internal::reallocate_aligned(data, size, ZFP_MEMORY_ALIGNMENT, bytes); bytes = size; codec.open(data, bytes); } @@ -173,7 +173,7 @@ class BlockStore { void deep_copy(const BlockStore& s) { free(); - zfp::clone_aligned(data, s.data, s.bytes, ZFP_MEMORY_ALIGNMENT); + zfp::internal::clone_aligned(data, s.data, s.bytes, ZFP_MEMORY_ALIGNMENT); bytes = s.bytes; index = s.index; codec = s.codec; @@ -185,7 +185,7 @@ class BlockStore { { free(); bytes = buffer_size(); - zfp::reallocate_aligned(data, bytes, ZFP_MEMORY_ALIGNMENT); + zfp::internal::reallocate_aligned(data, bytes, ZFP_MEMORY_ALIGNMENT); if (clear) std::fill(static_cast(data), static_cast(data) + bytes, uchar(0)); codec.open(data, bytes); @@ -195,7 +195,7 @@ class BlockStore { void free() { if (data) { - zfp::deallocate_aligned(data); + zfp::internal::deallocate_aligned(data); data = 0; bytes = 0; codec.close(); diff --git a/array/zfpcodec.h b/array/zfpcodec.h index e7e9bc279..c38f5da76 100644 --- a/array/zfpcodec.h +++ b/array/zfpcodec.h @@ -52,7 +52,7 @@ class zfp_base { size_t bz = (std::max(field->nz, size_t(1)) + 3) / 4; size_t bw = (std::max(field->nw, size_t(1)) + 3) / 4; size_t blocks = bx * by * bz * bw; - return zfp::round_up(blocks * stream->maxbits, stream_alignment()) / CHAR_BIT; + return zfp::internal::round_up(blocks * stream->maxbits, stream_alignment()) / CHAR_BIT; } // open bit stream diff --git a/array/zfpindex.h b/array/zfpindex.h index af59be6fe..815a0324b 100644 --- a/array/zfpindex.h +++ b/array/zfpindex.h @@ -108,7 +108,7 @@ class verbatim { void resize(size_t blocks) { this->blocks = blocks; - zfp::reallocate(data, capacity() * sizeof(*data)); + zfp::internal::reallocate(data, capacity() * sizeof(*data)); *data = 0; clear(); } @@ -146,7 +146,7 @@ class verbatim { // make a deep copy of index void deep_copy(const verbatim& index) { - zfp::clone(data, index.data, index.capacity()); + zfp::internal::clone(data, index.data, index.capacity()); blocks = index.blocks; block = index.block; } @@ -224,7 +224,7 @@ class hybrid4 { void resize(size_t blocks) { this->blocks = blocks; - zfp::reallocate(data, capacity() * sizeof(*data)); + zfp::internal::reallocate(data, capacity() * sizeof(*data)); clear(); } @@ -295,7 +295,7 @@ class hybrid4 { // make a deep copy of index void deep_copy(const hybrid4& index) { - zfp::clone(data, index.data, index.capacity()); + zfp::internal::clone(data, index.data, index.capacity()); blocks = index.blocks; block = index.block; ptr = index.ptr; @@ -382,7 +382,7 @@ class hybrid8 { void resize(size_t blocks) { this->blocks = blocks; - zfp::reallocate(data, capacity() * sizeof(*data)); + zfp::internal::reallocate(data, capacity() * sizeof(*data)); clear(); } @@ -457,7 +457,7 @@ class hybrid8 { // make a deep copy of index void deep_copy(const hybrid8& index) { - zfp::clone(data, index.data, index.capacity()); + zfp::internal::clone(data, index.data, index.capacity()); blocks = index.blocks; block = index.block; ptr = index.ptr; From dd915b95aa61655feb9a36a769e06deda01a7aed Mon Sep 17 00:00:00 2001 From: gmorriso Date: Mon, 28 Mar 2022 18:25:52 -0700 Subject: [PATCH 050/277] Update version files to support intermediate version numbers --- CMakeLists.txt | 12 +++++-- docs/source/high-level-api.rst | 27 ++++++++++++-- include/zfp/version.h | 65 +++++++++++++++++++++++++--------- zfp-config-version.cmake.in | 2 ++ 4 files changed, 86 insertions(+), 20 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 24ba7d657..ca5df5eb7 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -20,8 +20,16 @@ string(REGEX REPLACE ".*#define[ \t]+ZFP_VERSION_MINOR[ \t]+([0-9]+).*" "\\1" ZFP_VERSION_MINOR ${_zfp_h_contents}) string(REGEX REPLACE ".*#define[ \t]+ZFP_VERSION_PATCH[ \t]+([0-9]+).*" "\\1" ZFP_VERSION_PATCH ${_zfp_h_contents}) -set(ZFP_VERSION - "${ZFP_VERSION_MAJOR}.${ZFP_VERSION_MINOR}.${ZFP_VERSION_PATCH}") +string(REGEX REPLACE ".*#define[ \t]+ZFP_VERSION_TWEAK[ \t]+([0-9]+).*" + "\\1" ZFP_VERSION_TWEAK ${_zfp_h_contents}) + +if(${ZFP_VERSION_TWEAK} EQUAL 0) + set(ZFP_VERSION + "${ZFP_VERSION_MAJOR}.${ZFP_VERSION_MINOR}.${ZFP_VERSION_PATCH}") +else() + set(ZFP_VERSION + "${ZFP_VERSION_MAJOR}.${ZFP_VERSION_MINOR}.${ZFP_VERSION_PATCH}.${ZFP_VERSION_TWEAK}") +endif() project(ZFP VERSION ${ZFP_VERSION}) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 1500a1f70..d006baa3f 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -37,14 +37,17 @@ Macros .. c:macro:: ZFP_VERSION_MAJOR .. c:macro:: ZFP_VERSION_MINOR .. c:macro:: ZFP_VERSION_PATCH +.. c:macro:: ZFP_VERSION_TWEAK .. c:macro:: ZFP_VERSION .. c:macro:: ZFP_VERSION_STRING Macros identifying the |zfp| library version. :c:macro:`ZFP_VERSION` is - a single integer constructed from the previous three macros + a single integer constructed from the previous four macros (see :c:macro:`ZFP_MAKE_VERSION`). :c:macro:`ZFP_VERSION_STRING` is a string literal (see :c:macro:`ZFP_MAKE_VERSION_STRING`). See also - :c:data:`zfp_library_version` and :c:data:`zfp_version_string`. + :c:data:`zfp_library_version` and :c:data:`zfp_version_string`. + :c:macro:`ZFP_VERSION_TWEAK` is new as of |zfp| |versionrelease| and used + to mark intermediate develop versions. ---- @@ -59,6 +62,18 @@ Macros ---- +.. c:macro:: ZFP_MAKE_FULLVERSION(major, minor, patch, tweak) +.. c:macro:: ZFP_MAKE_FULLVERSION_STRING(major, minor, patch, tweak) + + Utility macros for constructing :c:macro:`ZFP_VERSION` and + :c:macro:`ZFP_VERSION_STRING`, respectively. Includes tweak version + used by intermediate develop versions. Available as of + |zfp| |versionrelease|, these macros may be used by applications to test + for a certain |zfp| version number, e.g., + :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(0, 5, 6, 1)`. + +---- + .. c:macro:: ZFP_CODEC Macro identifying the version of the compression CODEC. See also @@ -66,6 +81,14 @@ Macros ---- +.. c:macro:: ZFP_DEVELOP + + Macro identifying that the current version is an intermediate develop + version as opposed to an official release. Available as of |zfp| + |versionrelease|. + +---- + .. c:macro:: ZFP_MIN_BITS .. c:macro:: ZFP_MAX_BITS .. c:macro:: ZFP_MAX_PREC diff --git a/include/zfp/version.h b/include/zfp/version.h index 294cc151e..8a428614f 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -1,37 +1,70 @@ #ifndef ZFP_VERSION_H #define ZFP_VERSION_H +/* library version information */ +#define ZFP_VERSION_MAJOR 0 /* library major version number */ +#define ZFP_VERSION_MINOR 5 /* library minor version number */ +#define ZFP_VERSION_PATCH 5 /* library patch version number */ +#define ZFP_VERSION_TWEAK 0 /* library tweak version number */ + +#define ZFP_VERSION_RELEASE ZFP_VERSION_MAJOR + +/* codec version number (see also zfp_codec_version) */ +#define ZFP_CODEC 5 + +/* whether this is a full release or intermediate version */ +#define ZFP_DEVELOP 1 + /* stringification */ #define _zfp_str_(x) # x #define _zfp_str(x) _zfp_str_(x) /* macro for generating an integer version identifier */ -#define ZFP_MAKE_VERSION(major, minor, patch) \ - (((major) << 8) + \ - ((minor) << 4) + \ - ((patch) << 0)) +#define ZFP_MAKE_VERSION(major, minor, patch, tweak) \ + (((major) << 12) + \ + ((minor) << 8) + \ + ((patch) << 4) + \ + ((tweak) << 0)) -/* macro for generating a version string */ +/* macros for generating a version string */ #define ZFP_MAKE_VERSION_STRING(major, minor, patch) \ _zfp_str(major) "." \ _zfp_str(minor) "." \ _zfp_str(patch) -/* library version information */ -#define ZFP_VERSION_MAJOR 0 /* library major version number */ -#define ZFP_VERSION_MINOR 5 /* library minor version number */ -#define ZFP_VERSION_PATCH 5 /* library patch version number */ -#define ZFP_VERSION_RELEASE ZFP_VERSION_PATCH +#define ZFP_MAKE_VERSION_STRING(major, minor, patch) \ + _zfp_str(major) "." \ + _zfp_str(minor) "." \ + _zfp_str(patch) -/* codec version number (see also zfp_codec_version) */ -#define ZFP_CODEC 5 +#define ZFP_MAKE_FULLVERSION_STRING(major, minor, patch, tweak) \ + _zfp_str(major) "." \ + _zfp_str(minor) "." \ + _zfp_str(patch) "." \ + _zfp_str(tweak) + +#define ZFP_MAKE_FULLVERSION_STRING(major, minor, patch, tweak) \ + _zfp_str(major) "." \ + _zfp_str(minor) "." \ + _zfp_str(patch) "." \ + _zfp_str(tweak) /* library version number (see also zfp_library_version) */ -#define ZFP_VERSION \ - ZFP_MAKE_VERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) +#if ZFP_VERSION_TWEAK == 0 + #define ZFP_VERSION \ + ZFP_MAKE_VERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) +#else + #define ZFP_VERSION \ + ZFP_MAKE_FULLVERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH, ZFP_VERSION_TWEAK) +#endif /* library version string (see also zfp_version_string) */ -#define ZFP_VERSION_STRING \ - ZFP_MAKE_VERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) +#if ZFP_VERSION_TWEAK == 0 + #define ZFP_VERSION_STRING \ + ZFP_MAKE_VERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) +#else + #define ZFP_VERSION_STRING \ + ZFP_MAKE_FULLVERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH, ZFP_VERSION_TWEAK) +#endif #endif diff --git a/zfp-config-version.cmake.in b/zfp-config-version.cmake.in index 4a77db0a1..449327023 100644 --- a/zfp-config-version.cmake.in +++ b/zfp-config-version.cmake.in @@ -1,6 +1,8 @@ set(PACKAGE_VERSION_MAJOR @ZFP_VERSION_MAJOR@) set(PACKAGE_VERSION_MINOR @ZFP_VERSION_MINOR@) set(PACKAGE_VERSION_PATCH @ZFP_VERSION_PATCH@) +set(PACKAGE_VERSION_TWEAK @ZFP_VERSION_TWEAK@) + set(PACKAGE_VERSION @ZFP_VERSION@) # Check whether the requested PACKAGE_FIND_VERSION is compatible From 0335096e688ac5dbc6a50a12d357f039c55b21af Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 29 Mar 2022 13:56:53 -0700 Subject: [PATCH 051/277] Fix version.h compile error --- include/zfp/version.h | 28 ++++++---------------------- 1 file changed, 6 insertions(+), 22 deletions(-) diff --git a/include/zfp/version.h b/include/zfp/version.h index 8a428614f..61aa02c03 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -32,17 +32,6 @@ _zfp_str(minor) "." \ _zfp_str(patch) -#define ZFP_MAKE_VERSION_STRING(major, minor, patch) \ - _zfp_str(major) "." \ - _zfp_str(minor) "." \ - _zfp_str(patch) - -#define ZFP_MAKE_FULLVERSION_STRING(major, minor, patch, tweak) \ - _zfp_str(major) "." \ - _zfp_str(minor) "." \ - _zfp_str(patch) "." \ - _zfp_str(tweak) - #define ZFP_MAKE_FULLVERSION_STRING(major, minor, patch, tweak) \ _zfp_str(major) "." \ _zfp_str(minor) "." \ @@ -50,21 +39,16 @@ _zfp_str(tweak) /* library version number (see also zfp_library_version) */ -#if ZFP_VERSION_TWEAK == 0 - #define ZFP_VERSION \ - ZFP_MAKE_VERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) -#else - #define ZFP_VERSION \ - ZFP_MAKE_FULLVERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH, ZFP_VERSION_TWEAK) -#endif +#define ZFP_VERSION \ + ZFP_MAKE_VERSION(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH, ZFP_VERSION_TWEAK) /* library version string (see also zfp_version_string) */ #if ZFP_VERSION_TWEAK == 0 - #define ZFP_VERSION_STRING \ - ZFP_MAKE_VERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) + #define ZFP_VERSION_STRING \ + ZFP_MAKE_VERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH) #else - #define ZFP_VERSION_STRING \ - ZFP_MAKE_FULLVERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH, ZFP_VERSION_TWEAK) + #define ZFP_VERSION_STRING \ + ZFP_MAKE_FULLVERSION_STRING(ZFP_VERSION_MAJOR, ZFP_VERSION_MINOR, ZFP_VERSION_PATCH, ZFP_VERSION_TWEAK) #endif #endif From 5d3ac203e8ecab55631bbc46ac10e62ea49a22a7 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Apr 2022 10:42:21 -0700 Subject: [PATCH 052/277] Revert accidental change to ZFP_VERSION_RELEASE define --- include/zfp/version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/zfp/version.h b/include/zfp/version.h index 61aa02c03..784f1cf18 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -7,7 +7,7 @@ #define ZFP_VERSION_PATCH 5 /* library patch version number */ #define ZFP_VERSION_TWEAK 0 /* library tweak version number */ -#define ZFP_VERSION_RELEASE ZFP_VERSION_MAJOR +#define ZFP_VERSION_RELEASE ZFP_VERSION_PATCH /* codec version number (see also zfp_codec_version) */ #define ZFP_CODEC 5 From cf7bd9eac1d1ff9afb3346f186429ad27dd41d28 Mon Sep 17 00:00:00 2001 From: Jonas Haag Date: Wed, 13 Apr 2022 19:04:13 +0200 Subject: [PATCH 053/277] Fix exception handling in compress_numpy --- python/zfpy.pyx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/zfpy.pyx b/python/zfpy.pyx index 12b25cb72..6ac9e11c5 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -74,7 +74,7 @@ cpdef ztype_to_dtype(zfp_type ztype): except KeyError: raise ValueError("Unsupported zfp_type {}".format(ztype)) -cdef zfp_field* _init_field(np.ndarray arr): +cdef zfp_field* _init_field(np.ndarray arr) except NULL: shape = arr.shape cdef int ndim = arr.ndim cdef zfp_type ztype = dtype_to_ztype(arr.dtype) From 72fcac571d12fa75dde83e26e75631982ffb30ba Mon Sep 17 00:00:00 2001 From: gmorriso Date: Mon, 25 Apr 2022 15:46:59 -0700 Subject: [PATCH 054/277] Add cmake version checking to tests --- .github/workflows/cmake.yml | 52 ++++++++++++++++++++++++++++++++++++ tests/gitlab/corona-jobs.yml | 1 + tests/gitlab/gitlab-ci.yml | 2 ++ tests/gitlab/pascal-jobs.yml | 6 +++++ 4 files changed, 61 insertions(+) create mode 100644 .github/workflows/cmake.yml diff --git a/.github/workflows/cmake.yml b/.github/workflows/cmake.yml new file mode 100644 index 000000000..42cb615b4 --- /dev/null +++ b/.github/workflows/cmake.yml @@ -0,0 +1,52 @@ +# Confirms that zfp build generation runs correctly for the minimum required cmake version + +name: test-cmake +on: push +jobs: + build: + runs-on: ${{matrix.os}} + strategy: + fail-fast: false + matrix: + include: + - os: ubuntu-latest + cxx_compiler: g++-10 + c_compiler: gcc-10 + target: all + + - os: windows-latest + cxx_compiler: clang++ + c_compiler: clang + target: all + + steps: + - uses: actions/checkout@v2 + + - name: Setup CMake + uses: jwlawson/actions-setup-cmake@v1.12 + with: + cmake-version: '3.9.x' + + - name: Setup Python + uses: actions/setup-python@v2 + with: + python-version: '3.x' + architecture: x64 + + - name: Install zfpy dependencies + run: | + python -m pip install cython + python -m pip install numpy + + - name: Setup MSBuild (Windows) + id: msbuild + if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} + uses: microsoft/setup-msbuild@v1.0.3 + + - name: Run CMake + id: cmake + run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=ON -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DBUILD_ZFORP=ON + + - name: Build + id: build + run: cmake --build ${{github.workspace}}/build --target ${{matrix.target}} --config Release diff --git a/tests/gitlab/corona-jobs.yml b/tests/gitlab/corona-jobs.yml index 06acdc85f..fe75a6e54 100644 --- a/tests/gitlab/corona-jobs.yml +++ b/tests/gitlab/corona-jobs.yml @@ -4,6 +4,7 @@ rocm-3.10.0_build: variables: + ci_cmake: "cmake/3.21.1" ci_cmp_mod: "rocm/3.10.0" ci_cmp_path: "/opt/rocm-3.10.0/hip" extends: [.hip, .corona_build_gpu] diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 7594eb402..e7dc30654 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -27,6 +27,7 @@ stages: .build_cpu: before_script: - module reset + - module load $ci_cmake - module load $ci_cmp_mod - |- if [ "$ci_lang" == "cpp" ]; then @@ -57,6 +58,7 @@ stages: before_script: - module reset - module load opt + - module load $ci_cmake - module load $ci_cmp_mod script: - mkdir build diff --git a/tests/gitlab/pascal-jobs.yml b/tests/gitlab/pascal-jobs.yml index bfb968808..8bbcaf155 100644 --- a/tests/gitlab/pascal-jobs.yml +++ b/tests/gitlab/pascal-jobs.yml @@ -4,6 +4,7 @@ cuda-10.1.168_build: variables: + ci_cmake: "cmake/3.9.2" ci_cmp_mod: "cuda/10.1.168" extends: [.cuda, .pascal_build_gpu] needs: [] @@ -20,6 +21,7 @@ cuda-10.1.168_test: cpp_gnu-7.3.0_build: variables: + ci_cmake: "cmake/3.9.2" ci_cxx_cmp: "g++" ci_c_cmp: "gcc" ci_cmp_mod: "gcc/7.3.0" @@ -33,6 +35,7 @@ cpp_gnu-7.3.0_test: cpp_clang-10.0.0_build: variables: + ci_cmake: "cmake/3.9.2" ci_cxx_cmp: "clang++" ci_c_cmp: "clang" ci_cmp_mod: "clang/10.0.0" @@ -46,6 +49,7 @@ cpp_clang-10.0.0_test: cpp_intel-19.0.4_build: variables: + ci_cmake: "cmake/3.9.2" ci_cxx_cmp: "icpc" ci_c_cmp: "icc" ci_cmp_mod: "intel/19.0.4" @@ -59,6 +63,7 @@ cpp_intel-19.0.4_test: cpp_pgi-21.1_build: variables: + ci_cmake: "cmake/3.9.2" ci_cxx_cmp: "pgc++" ci_c_cmp: "pgcc" ci_cmp_mod: "pgi/21.1" @@ -76,6 +81,7 @@ cpp_pgi-21.1_test: c_gnu-7.3.0_build: variables: + ci_cmake: "cmake/3.9.2" ci_c_cmp: "gcc" ci_cmp_mod: "gcc/7.3.0" extends: [.c, .pascal_build_cpu] From 9dcf734c8c9195602fa70596f543ebfe234c51ea Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 29 Apr 2022 11:10:23 -0700 Subject: [PATCH 055/277] temporarily disable actions side of cmake test --- .github/workflows/cmake.yml | 100 ++++++++++++++++++------------------ 1 file changed, 50 insertions(+), 50 deletions(-) diff --git a/.github/workflows/cmake.yml b/.github/workflows/cmake.yml index 42cb615b4..abd4a7b29 100644 --- a/.github/workflows/cmake.yml +++ b/.github/workflows/cmake.yml @@ -1,52 +1,52 @@ # Confirms that zfp build generation runs correctly for the minimum required cmake version -name: test-cmake -on: push -jobs: - build: - runs-on: ${{matrix.os}} - strategy: - fail-fast: false - matrix: - include: - - os: ubuntu-latest - cxx_compiler: g++-10 - c_compiler: gcc-10 - target: all - - - os: windows-latest - cxx_compiler: clang++ - c_compiler: clang - target: all - - steps: - - uses: actions/checkout@v2 - - - name: Setup CMake - uses: jwlawson/actions-setup-cmake@v1.12 - with: - cmake-version: '3.9.x' - - - name: Setup Python - uses: actions/setup-python@v2 - with: - python-version: '3.x' - architecture: x64 - - - name: Install zfpy dependencies - run: | - python -m pip install cython - python -m pip install numpy - - - name: Setup MSBuild (Windows) - id: msbuild - if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} - uses: microsoft/setup-msbuild@v1.0.3 - - - name: Run CMake - id: cmake - run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=ON -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DBUILD_ZFORP=ON - - - name: Build - id: build - run: cmake --build ${{github.workspace}}/build --target ${{matrix.target}} --config Release +#name: test-cmake +#on: push +#jobs: +# build: +# runs-on: ${{matrix.os}} +# strategy: +# fail-fast: false +# matrix: +# include: +# - os: ubuntu-latest +# cxx_compiler: g++-10 +# c_compiler: gcc-10 +# target: all +# +# - os: windows-latest +# cxx_compiler: clang++ +# c_compiler: clang +# target: all +# +# steps: +# - uses: actions/checkout@v2 +# +# - name: Setup CMake +# uses: jwlawson/actions-setup-cmake@v1.12 +# with: +# cmake-version: '3.9.x' +# +# - name: Setup Python +# uses: actions/setup-python@v2 +# with: +# python-version: '3.x' +# architecture: x64 +# +# - name: Install zfpy dependencies +# run: | +# python -m pip install cython +# python -m pip install numpy +# +# - name: Setup MSBuild (Windows) +# id: msbuild +# if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} +# uses: microsoft/setup-msbuild@v1.0.3 +# +# - name: Run CMake +# id: cmake +# run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=ON -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DBUILD_ZFORP=ON +# +# - name: Build +# id: build +# run: cmake --build ${{github.workspace}}/build --target ${{matrix.target}} --config Release From b40ab53ddc8ffe353152ead9f32b0c5f3ee8bcdc Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 29 Apr 2022 11:11:54 -0700 Subject: [PATCH 056/277] temporarily disable actions side of cmake test --- .github/workflows/cmake.yml | 52 ------------------------------------- 1 file changed, 52 deletions(-) delete mode 100644 .github/workflows/cmake.yml diff --git a/.github/workflows/cmake.yml b/.github/workflows/cmake.yml deleted file mode 100644 index abd4a7b29..000000000 --- a/.github/workflows/cmake.yml +++ /dev/null @@ -1,52 +0,0 @@ -# Confirms that zfp build generation runs correctly for the minimum required cmake version - -#name: test-cmake -#on: push -#jobs: -# build: -# runs-on: ${{matrix.os}} -# strategy: -# fail-fast: false -# matrix: -# include: -# - os: ubuntu-latest -# cxx_compiler: g++-10 -# c_compiler: gcc-10 -# target: all -# -# - os: windows-latest -# cxx_compiler: clang++ -# c_compiler: clang -# target: all -# -# steps: -# - uses: actions/checkout@v2 -# -# - name: Setup CMake -# uses: jwlawson/actions-setup-cmake@v1.12 -# with: -# cmake-version: '3.9.x' -# -# - name: Setup Python -# uses: actions/setup-python@v2 -# with: -# python-version: '3.x' -# architecture: x64 -# -# - name: Install zfpy dependencies -# run: | -# python -m pip install cython -# python -m pip install numpy -# -# - name: Setup MSBuild (Windows) -# id: msbuild -# if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} -# uses: microsoft/setup-msbuild@v1.0.3 -# -# - name: Run CMake -# id: cmake -# run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=ON -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") -DBUILD_ZFORP=ON -# -# - name: Build -# id: build -# run: cmake --build ${{github.workspace}}/build --target ${{matrix.target}} --config Release From 9a1e3409f9ee4a284e679bb0865c22ea676ef1ae Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 15 Jun 2022 19:17:01 -0700 Subject: [PATCH 057/277] Introduce bitstream_{offset,size,count} types for 64-bit support --- CHANGELOG.md | 3 + array/gencodec.h | 38 +++--- array/zfpcodec.h | 36 +++--- array/zfpindex.h | 68 +++++----- docs/source/bit-stream.rst | 68 ++++++---- docs/source/codec.inc | 36 +++--- docs/source/index.inc | 4 +- include/bitstream.h | 33 +++-- src/cuda_zfp/cuZFP.cu | 4 +- src/inline/bitstream.c | 137 +++++++++++---------- src/share/parallel.c | 4 +- src/template/decode.c | 4 +- src/template/encode.c | 4 +- tests/src/inline/testBitstream.c | 82 ++++++------ tests/src/inline/testBitstreamSmallWsize.c | 6 +- tests/src/inline/testBitstreamStrided.c | 8 +- 16 files changed, 291 insertions(+), 244 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fd7e175da..f4b54a81b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,9 @@ few of the API changes, other than to cfp, should impact existing code. - `size_t` and `ptrdiff_t` replace `uint` and `int` for array sizes and strides in the array classes and C API. - `zfp_bool` replaces `int` as Boolean type in the C API. +- `bitstream_offset` and `bitstream_size` replace `size_t` to ensure support + for 64-bit offsets into and lengths of bit streams. Consequently, the + `bitstream` API has changed accordingly. - All array and view iterators are now random-access iterators. - Array inspectors now return `const_reference` rather than a scalar type like `float` to allow obtaining a `const_pointer` to an element diff --git a/array/gencodec.h b/array/gencodec.h index ebb727fa0..7557c2388 100644 --- a/array/gencodec.h +++ b/array/gencodec.h @@ -148,7 +148,7 @@ class generic_base { protected: // pointer to beginning of block - InternalType* begin(size_t offset) const + InternalType* begin(bitstream_offset offset) const { if (offset % internal_size_bits) throw zfp::exception("zfp generic codec bit offset alignment error"); @@ -156,7 +156,7 @@ class generic_base { } // store full contiguous block to memory - size_t encode_block(size_t offset, const ExternalType* block) const + size_t encode_block(bitstream_offset offset, const ExternalType* block) const { InternalType* ptr = begin(offset); for (size_t n = block_size; n--;) @@ -165,7 +165,7 @@ class generic_base { } // load full contiguous block from memory - size_t decode_block(size_t offset, ExternalType* block) const + size_t decode_block(bitstream_offset offset, ExternalType* block) const { const InternalType* ptr = begin(offset); for (size_t n = block_size; n--;) @@ -187,14 +187,14 @@ template class generic1 : public generic_base<1, ExternalType, InternalType> { public: // encode contiguous 1D block - size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + size_t encode_block(bitstream_offset offset, uint shape, const ExternalType* block) const { return shape ? encode_block_strided(offset, shape, block, 1) : encode_block(offset, block); } // encode 1D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx) const { InternalType* q = begin(offset); size_t nx = 4; @@ -207,14 +207,14 @@ class generic1 : public generic_base<1, ExternalType, InternalType> { } // decode contiguous 1D block - size_t decode_block(size_t offset, uint shape, ExternalType* block) const + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const { return shape ? decode_block_strided(offset, shape, block, 1) : decode_block(offset, block); } // decode 1D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx) const + size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx) const { const InternalType* q = begin(offset); size_t nx = 4; @@ -238,14 +238,14 @@ template class generic2 : public generic_base<2, ExternalType, InternalType> { public: // encode contiguous 2D block - size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + size_t encode_block(bitstream_offset offset, uint shape, const ExternalType* block) const { return shape ? encode_block_strided(offset, shape, block, 1, 4) : encode_block(offset, block); } // encode 2D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const { InternalType* q = begin(offset); size_t nx = 4; @@ -261,14 +261,14 @@ class generic2 : public generic_base<2, ExternalType, InternalType> { } // decode contiguous 2D block - size_t decode_block(size_t offset, uint shape, ExternalType* block) const + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const { return shape ? decode_block_strided(offset, shape, block, 1, 4) : decode_block(offset, block); } // decode 2D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const + size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const { const InternalType* q = begin(offset); size_t nx = 4; @@ -295,14 +295,14 @@ template class generic3 : public generic_base<3, ExternalType, InternalType> { public: // encode contiguous 3D block - size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + size_t encode_block(bitstream_offset offset, uint shape, const ExternalType* block) const { return shape ? encode_block_strided(offset, shape, block, 1, 4, 16) : encode_block(offset, block); } // encode 3D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const { InternalType* q = begin(offset); size_t nx = 4; @@ -321,14 +321,14 @@ class generic3 : public generic_base<3, ExternalType, InternalType> { } // decode contiguous 3D block - size_t decode_block(size_t offset, uint shape, ExternalType* block) const + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const { return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) : decode_block(offset, block); } // decode 3D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const { const InternalType* q = begin(offset); size_t nx = 4; @@ -358,14 +358,14 @@ template class generic4 : public generic_base<4, ExternalType, InternalType> { public: // encode contiguous 4D block - size_t encode_block(size_t offset, uint shape, const ExternalType* block) const + size_t encode_block(bitstream_offset offset, uint shape, const ExternalType* block) const { return shape ? encode_block_strided(offset, shape, block, 1, 4, 16, 64) : encode_block(offset, block); } // encode 4D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const { InternalType* q = begin(offset); size_t nx = 4; @@ -387,14 +387,14 @@ class generic4 : public generic_base<4, ExternalType, InternalType> { } // decode contiguous 4D block - size_t decode_block(size_t offset, uint shape, ExternalType* block) const + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const { return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) : decode_block(offset, block); } // decode 4D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const { const InternalType* q = begin(offset); size_t nx = 4; diff --git a/array/zfpcodec.h b/array/zfpcodec.h index e7e9bc279..17611ad22 100644 --- a/array/zfpcodec.h +++ b/array/zfpcodec.h @@ -127,7 +127,7 @@ class zfp_base { } // encode full contiguous block - size_t encode_block(size_t offset, const Scalar* block) const + size_t encode_block(bitstream_offset offset, const Scalar* block) const { stream_wseek(stream->stream, offset); size_t size = zfp::encode_block(stream, block); @@ -136,7 +136,7 @@ class zfp_base { } // decode full contiguous block - size_t decode_block(size_t offset, Scalar* block) const + size_t decode_block(bitstream_offset offset, Scalar* block) const { stream_rseek(stream->stream, offset); size_t size = zfp::decode_block(stream, block); @@ -152,14 +152,14 @@ template class zfp1 : public zfp_base<1, Scalar> { public: // encode contiguous 1D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) const + size_t encode_block(bitstream_offset offset, uint shape, const Scalar* block) const { return shape ? encode_block_strided(offset, shape, block, 1) : encode_block(offset, block); } // encode 1D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx) const { size_t size; stream_wseek(stream->stream, offset); @@ -174,14 +174,14 @@ class zfp1 : public zfp_base<1, Scalar> { } // decode contiguous 1D block - size_t decode_block(size_t offset, uint shape, Scalar* block) const + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const { return shape ? decode_block_strided(offset, shape, block, 1) : decode_block(offset, block); } // decode 1D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx) const + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx) const { size_t size; stream_rseek(stream->stream, offset); @@ -206,14 +206,14 @@ template class zfp2 : public zfp_base<2, Scalar> { public: // encode contiguous 2D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) const + size_t encode_block(bitstream_offset offset, uint shape, const Scalar* block) const { return shape ? encode_block_strided(offset, shape, block, 1, 4) : encode_block(offset, block); } // encode 2D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const { size_t size; stream_wseek(stream->stream, offset); @@ -229,14 +229,14 @@ class zfp2 : public zfp_base<2, Scalar> { } // decode contiguous 2D block - size_t decode_block(size_t offset, uint shape, Scalar* block) const + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const { return shape ? decode_block_strided(offset, shape, block, 1, 4) : decode_block(offset, block); } // decode 2D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const { size_t size; stream_rseek(stream->stream, offset); @@ -262,14 +262,14 @@ template class zfp3 : public zfp_base<3, Scalar> { public: // encode contiguous 3D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) const + size_t encode_block(bitstream_offset offset, uint shape, const Scalar* block) const { return shape ? encode_block_strided(offset, shape, block, 1, 4, 16) : encode_block(offset, block); } // encode 3D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const { size_t size; stream_wseek(stream->stream, offset); @@ -286,14 +286,14 @@ class zfp3 : public zfp_base<3, Scalar> { } // decode contiguous 3D block - size_t decode_block(size_t offset, uint shape, Scalar* block) const + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const { return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) : decode_block(offset, block); } // decode 3D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const { size_t size; stream_rseek(stream->stream, offset); @@ -320,14 +320,14 @@ template class zfp4 : public zfp_base<4, Scalar> { public: // encode contiguous 4D block - size_t encode_block(size_t offset, uint shape, const Scalar* block) const + size_t encode_block(bitstream_offset offset, uint shape, const Scalar* block) const { return shape ? encode_block_strided(offset, shape, block, 1, 4, 16, 64) : encode_block(offset, block); } // encode 4D block from strided storage - size_t encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const { size_t size; stream_wseek(stream->stream, offset); @@ -345,14 +345,14 @@ class zfp4 : public zfp_base<4, Scalar> { } // decode contiguous 4D block - size_t decode_block(size_t offset, uint shape, Scalar* block) const + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const { return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) : decode_block(offset, block); } // decode 4D block to strided storage - size_t decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const { size_t size; stream_rseek(stream->stream, offset); diff --git a/array/zfpindex.h b/array/zfpindex.h index af59be6fe..0f3fc44e5 100644 --- a/array/zfpindex.h +++ b/array/zfpindex.h @@ -29,13 +29,13 @@ class implicit { } // range of offsets spanned by indexed data in bits - size_t range() const { return block_offset(blocks); } + bitstream_size range() const { return block_offset(blocks); } // bit size of given block size_t block_size(size_t /*block_index*/) const { return bits_per_block; } // bit offset of given block - size_t block_offset(size_t block_index) const { return bits_per_block * block_index; } + bitstream_offset block_offset(size_t block_index) const { return block_index * bits_per_block; } // reset index void clear() {} @@ -93,13 +93,13 @@ class verbatim { } // range of offsets spanned by indexed data in bits - size_t range() const { return block_offset(blocks); } + bitstream_size range() const { return block_offset(blocks); } // bit size of given block - size_t block_size(size_t block_index) const { return block_offset(block_index + 1) - block_offset(block_index); } + size_t block_size(size_t block_index) const { return static_cast(block_offset(block_index + 1) - block_offset(block_index)); } // bit offset of given block - size_t block_offset(size_t block_index) const { return static_cast(data[block_index]); } + bitstream_offset block_offset(size_t block_index) const { return static_cast(data[block_index]); } // reset index void clear() { block = 0; } @@ -189,7 +189,7 @@ class hybrid4 { } // range of offsets spanned by indexed data in bits - size_t range() const { return end; } + bitstream_size range() const { return end; } // bit size of given block size_t block_size(size_t block_index) const @@ -197,12 +197,12 @@ class hybrid4 { size_t chunk = block_index / 4; size_t which = block_index % 4; return which == 3u - ? block_offset(block_index + 1) - block_offset(block_index) - : data[chunk].lo[which + 1] - data[chunk].lo[which]; + ? static_cast(block_offset(block_index + 1) - block_offset(block_index)) + : static_cast(data[chunk].lo[which + 1] - data[chunk].lo[which]); } // bit offset of given block - size_t block_offset(size_t block_index) const + bitstream_offset block_offset(size_t block_index) const { // if index is being built, point offset to end if (block_index == block) @@ -210,7 +210,7 @@ class hybrid4 { // index has already been built; decode offset size_t chunk = block_index / 4; size_t which = block_index % 4; - return (size_t(data[chunk].hi) << shift) + data[chunk].lo[which]; + return (bitstream_offset(data[chunk].hi) << shift) + data[chunk].lo[which]; } // reset index @@ -264,12 +264,12 @@ class hybrid4 { size_t which = block % 4; buffer[which] = size; if (which == 3u) { - // chunk is complete; encode it (double shift in case ptr is 32 bits) - if (((ptr >> 16) >> 16) >> shift) + // chunk is complete; encode it + if (ptr >> (32 + shift)) throw zfp::exception("zfp block offset is too large for hybrid4 index"); // store high bits data[chunk].hi = static_cast(ptr >> shift); - size_t base = size_t(data[chunk].hi) << shift; + bitstream_offset base = bitstream_offset(data[chunk].hi) << shift; // store low bits for (uint k = 0; k < 4; k++) { data[chunk].lo[k] = static_cast(ptr - base); @@ -305,12 +305,12 @@ class hybrid4 { static const uint shift = 12; // number of bits to shift hi bits - record* data; // block offset array - size_t blocks; // number of blocks - size_t block; // current block index - size_t end; // offset to last block - size_t ptr; // offset to current chunk of blocks - size_t buffer[4]; // buffer of 4 blocks to be stored together + record* data; // block offset array + size_t blocks; // number of blocks + size_t block; // current block index + bitstream_offset end; // offset to last block + bitstream_offset ptr; // offset to current chunk of blocks + size_t buffer[4]; // bit sizes 4 blocks to be stored together }; // hybrid block index (8 blocks/chunk; 16 bits/block; 86-14dims bit offsets) -- @@ -347,7 +347,7 @@ class hybrid8 { } // range of offsets spanned by indexed data in bits - size_t range() const { return end; } + bitstream_size range() const { return end; } // bit size of given block size_t block_size(size_t block_index) const @@ -355,12 +355,12 @@ class hybrid8 { size_t chunk = block_index / 8; size_t which = block_index % 8; return which == 7u - ? block_offset(block_index + 1) - block_offset(block_index) - : static_cast(size(data[2 * chunk + 0], data[2 * chunk + 1], static_cast(which))); + ? static_cast(block_offset(block_index + 1) - block_offset(block_index)) + : size(data[2 * chunk + 0], data[2 * chunk + 1], static_cast(which)); } // bit offset of given block - size_t block_offset(size_t block_index) const + bitstream_offset block_offset(size_t block_index) const { // if index is being built, point offset to end if (block_index == block) @@ -368,7 +368,7 @@ class hybrid8 { // index has already been built; decode offset size_t chunk = block_index / 8; size_t which = block_index % 8; - return static_cast(offset(data[2 * chunk + 0], data[2 * chunk + 1], static_cast(which))); + return offset(data[2 * chunk + 0], data[2 * chunk + 1], static_cast(which)); } // reset index @@ -466,17 +466,17 @@ class hybrid8 { } // kth size in chunk, 0 <= k <= 6 - static uint64 size(uint64 h, uint64 l, uint k) + static size_t size(uint64 h, uint64 l, uint k) { // extract high and low bits h >>= (6 - k) * hbits; h &= (UINT64C(1) << hbits) - 1; l >>= (6 - k) * lbits; l &= (UINT64C(1) << lbits) - 1; // combine base offset with high and low bits - return (h << lbits) + l; + return static_cast((h << lbits) + l); } // kth offset in chunk, 0 <= k <= 7 - static uint64 offset(uint64 h, uint64 l, uint k) + static bitstream_offset offset(uint64 h, uint64 l, uint k) { // extract all but lowest (8 * hbits) bits uint64 base = h >> (8 * hbits); @@ -485,7 +485,7 @@ class hybrid8 { h = hsum(h >> ((7 - k) * hbits)); l = lsum(l >> ((7 - k) * lbits)); // combine base offset with high and low bits - return (((base << hbits) + h) << lbits) + l; + return static_cast((((base << hbits) + h) << lbits) + l); } // sum of (up to) eight packed 8-bit numbers (efficient version of sum8) @@ -522,12 +522,12 @@ class hybrid8 { static const uint lbits = 8; // 64 bits partitioned into 8 static const uint hbits = 2 * (dims - 1); // log2(4^d * maxprec / 2^lbits) - uint64* data; // block offset array - size_t blocks; // number of blocks - size_t block; // current block index - size_t end; // offset to last block - size_t ptr; // offset to current set of blocks - size_t buffer[8]; // buffer of 8 blocks to be stored together + uint64* data; // block offset array + size_t blocks; // number of blocks + size_t block; // current block index + bitstream_offset end; // offset to last block + bitstream_offset ptr; // offset to current set of blocks + size_t buffer[8]; // sizes of 8 blocks to be stored together }; } // index diff --git a/docs/source/bit-stream.rst b/docs/source/bit-stream.rst index 9dd2187e2..e68640238 100644 --- a/docs/source/bit-stream.rst +++ b/docs/source/bit-stream.rst @@ -66,7 +66,7 @@ section. Types ----- -.. c:type:: stream_word +.. c:type:: bitstream_word Bits are buffered and read/written in units of words. By default, the bit stream word type is 64 bits, but may be set to 8, 16, or 32 bits @@ -82,6 +82,31 @@ Types ---- +.. c:type:: bitstream_offset + + Type holding the offset, measured in number of bits, into the bit stream + where the next bit will be read or written. This type allows referencing + bits in streams at least 2\ :sup:`64` bits long. Note that it is possible + that :code:`sizeof(bitstream_offset) > sizeof(size_t)` since a stream may + be as long as `sizeof(size_t) * CHAR_BIT` bits. + +---- + +.. c:type:: bitstream_size + + Alias for :c:type:`bitstream_offset` that signifies the bit length of a + stream or substream rather than an offset into it. + +---- + +.. c:type:: bitstream_count + + Type sufficient to count the number of bits read or written in functions + like :c:func:`stream_read_bits` and :c:func:`stream_write_bits`. + :code:`sizeof(bitstream_count) <= sizeof(bitstream_size)`. + +---- + .. c:type:: bitstream The bit stream struct maintains all the state associated with a bit @@ -90,13 +115,13 @@ Types :: struct bitstream { - uint bits; // number of buffered bits (0 <= bits < word size) - stream_word buffer; // buffer for incoming/outgoing bits (buffer < 2^bits) - stream_word* ptr; // pointer to next word to be read/written - stream_word* begin; // beginning of stream - stream_word* end; // end of stream (currently unused) - size_t mask; // one less the block size in number of words (if BIT_STREAM_STRIDED) - ptrdiff_t delta; // number of words between consecutive blocks (if BIT_STREAM_STRIDED) + bitstream_count bits; // number of buffered bits (0 <= bits < word size) + bitstream_word buffer; // incoming/outgoing bits (buffer < 2^bits) + bitstream_word* ptr; // pointer to next word to be read/written + bitstream_word* begin; // beginning of stream + bitstream_word* end; // end of stream (not enforced) + size_t mask; // one less the block size in number of words (if BIT_STREAM_STRIDED) + ptrdiff_t delta; // number of words between consecutive blocks (if BIT_STREAM_STRIDED) }; .. _bs-data: @@ -134,7 +159,7 @@ Functions ---- -.. c:function:: size_t stream_alignment() +.. c:function:: bitstream_count stream_alignment() Word size in bits. This is a functional form of the constant :c:var:`stream_word_bits` and returns the same value. @@ -174,26 +199,26 @@ Functions ---- -.. c:function:: uint64 stream_read_bits(bitstream* stream, uint n) +.. c:function:: uint64 stream_read_bits(bitstream* stream, bitstream_count n) Read and return 0 |leq| *n* |leq| 64 bits from *stream*. ---- -.. c:function:: uint64 stream_write_bits(bitstream* stream, uint64 value, uint n) +.. c:function:: uint64 stream_write_bits(bitstream* stream, uint64 value, bitstream_count n) Write 0 |leq| *n* |leq| 64 low bits of *value* to *stream*. Return any remaining bits from *value*, i.e., *value* >> *n*. ---- -.. c:function:: size_t stream_rtell(const bitstream* stream) +.. c:function:: bitstream_offset stream_rtell(const bitstream* stream) Return bit offset to next bit to be read. ---- -.. c:function:: size_t stream_wtell(const bitstream* stream) +.. c:function:: bitstream_offset stream_wtell(const bitstream* stream) Return bit offset to next bit to be written. @@ -206,33 +231,33 @@ Functions ---- -.. c:function:: void stream_rseek(bitstream* stream, size_t offset) +.. c:function:: void stream_rseek(bitstream* stream, bitstream_offset offset) Position stream for reading at given bit offset. This places the stream in read mode. ---- -.. c:function:: void stream_wseek(bitstream* stream, size_t offset) +.. c:function:: void stream_wseek(bitstream* stream, bitstream_offset offset) Position stream for writing at given bit offset. This places the stream in write mode. ---- -.. c:function:: void stream_skip(bitstream* stream, uint n) +.. c:function:: void stream_skip(bitstream* stream, bitstream_count n) Skip over the next *n* bits, i.e., without reading them. ---- -.. c:function:: void stream_pad(bitstream* stream, uint n) +.. c:function:: void stream_pad(bitstream* stream, bitstream_count n) Append *n* zero-bits to *stream*. ---- -.. c:function:: size_t stream_align(bitstream* stream) +.. c:function:: bitstream_count stream_align(bitstream* stream) Align stream on next word boundary by skipping bits. No skipping is done if the stream is already word aligned. Return the number of @@ -240,7 +265,7 @@ Functions ---- -.. c:function:: size_t stream_flush(bitstream* stream) +.. c:function:: bitstream_count stream_flush(bitstream* stream) Write out any remaining buffered bits. When one or more bits are buffered, append zero-bits to the stream to align it on a word boundary. @@ -248,7 +273,7 @@ Functions ---- -.. c:function:: void stream_copy(bitstream* dst, bitstream* src, size_t n) +.. c:function:: void stream_copy(bitstream* dst, bitstream* src, bitstream_size n) Copy *n* bits from *src* to *dst*, advancing both bit streams. @@ -272,4 +297,5 @@ Functions .. c:function:: int stream_set_stride(bitstream* stream, size_t block, ptrdiff_t delta) Set block size, *block*, in number of words and spacing, *delta*, in number - of blocks for strided access. Requires :c:macro:`BIT_STREAM_STRIDED`. + of blocks for strided access. Return nonzero upon success. Requires + :c:macro:`BIT_STREAM_STRIDED`. diff --git a/docs/source/codec.inc b/docs/source/codec.inc index 5b4c0b67b..a84343dfd 100644 --- a/docs/source/codec.inc +++ b/docs/source/codec.inc @@ -172,7 +172,7 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- -.. cpp:function:: size_t codec::encode_block(size_t offset, const Scalar* block) const +.. cpp:function:: size_t codec::encode_block(bitstream_offset offset, const Scalar* block) const Encode contiguous *block* of |4powd| scalars and store at specified bit *offset* within compressed-data buffer. Return the number of bits of @@ -183,7 +183,7 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- -.. cpp:function:: size_t codec::decode_block(size_t offset, Scalar* block) const +.. cpp:function:: size_t codec::decode_block(bitstream_offset offset, Scalar* block) const Decode contiguous *block* of |4powd| scalars from specified bit *offset* within compressed-data buffer (see :cpp:func:`codec::encode_block`). @@ -192,10 +192,10 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- -.. cpp:function:: size_t codec1::encode_block(size_t offset, uint shape, const Scalar* block) const -.. cpp:function:: size_t codec2::encode_block(size_t offset, uint shape, const Scalar* block) const -.. cpp:function:: size_t codec3::encode_block(size_t offset, uint shape, const Scalar* block) const -.. cpp:function:: size_t codec4::encode_block(size_t offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec1::encode_block(bitstream_offset offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec2::encode_block(bitstream_offset offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec3::encode_block(bitstream_offset offset, uint shape, const Scalar* block) const +.. cpp:function:: size_t codec4::encode_block(bitstream_offset offset, uint shape, const Scalar* block) const Encode contiguous *block* of data of given *shape* and store at specified bit *offset* within compressed-data buffer. Return the number of bits of @@ -210,10 +210,10 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- -.. cpp:function:: size_t codec1::decode_block(size_t offset, uint shape, Scalar* block) const -.. cpp:function:: size_t codec2::decode_block(size_t offset, uint shape, Scalar* block) const -.. cpp:function:: size_t codec3::decode_block(size_t offset, uint shape, Scalar* block) const -.. cpp:function:: size_t codec4::decode_block(size_t offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec1::decode_block(bitstream_offset offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec2::decode_block(bistream_offset offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec3::decode_block(bitstream_offset offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec4::decode_block(bitstream_offset offset, uint shape, Scalar* block) const Decode contiguous *block* of data of given *shape* from specified bit *offset* within compressed-data buffer (see also @@ -223,10 +223,10 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- -.. cpp:function:: size_t codec1::encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx) const -.. cpp:function:: size_t codec2::encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const -.. cpp:function:: size_t codec3::encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const -.. cpp:function:: size_t codec4::encode_block_strided(size_t offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const +.. cpp:function:: size_t codec1::encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx) const +.. cpp:function:: size_t codec2::encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const +.. cpp:function:: size_t codec3::encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const +.. cpp:function:: size_t codec4::encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const Encode block of data stored at *p* with strides *sx*, *sy*, *sz*, and *sw*. See :c:type:`zfp_field` for information on strided storage. @@ -235,10 +235,10 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- -.. cpp:function:: size_t codec1::decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx) const -.. cpp:function:: size_t codec2::decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const -.. cpp:function:: size_t codec3::decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const -.. cpp:function:: size_t codec4::decode_block_strided(size_t offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const +.. cpp:function:: size_t codec1::decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx) const +.. cpp:function:: size_t codec2::decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const +.. cpp:function:: size_t codec3::decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const +.. cpp:function:: size_t codec4::decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const Decode block to strided storage pointed to by *p* with strides *sx*, *sy*, *sz*, and *sw*. See :c:type:`zfp_field` for information on strided storage. diff --git a/docs/source/index.inc b/docs/source/index.inc index 40e354eb1..cf6ba9536 100644 --- a/docs/source/index.inc +++ b/docs/source/index.inc @@ -95,7 +95,7 @@ interface with the |zfp| compressed-array classes. ---- -.. cpp:function:: size_t index::range() const +.. cpp:function:: bitstream_size index::range() const Range of bit offsets spanned by index. This equals the total number of bits of compressed-array data. @@ -108,7 +108,7 @@ interface with the |zfp| compressed-array classes. ---- -.. cpp:function:: size_t index::block_offset(size_t block_index) const +.. cpp:function:: bitstream_offset index::block_offset(size_t block_index) const Bit offset to compressed block data. diff --git a/include/bitstream.h b/include/bitstream.h index 8ba1685d4..f9b292186 100644 --- a/include/bitstream.h +++ b/include/bitstream.h @@ -8,6 +8,15 @@ /* forward declaration of opaque type */ typedef struct bitstream bitstream; +/* bit offset into stream where bits are read/written */ +typedef uint64 bitstream_offset; + +/* type for counting number of bits in a stream */ +typedef bitstream_offset bitstream_size; + +/* type for counting a small number of bits in a stream */ +typedef size_t bitstream_count; + extern_ const size_t stream_word_bits; /* bit stream granularity */ #ifndef inline_ @@ -25,7 +34,7 @@ void stream_close(bitstream* stream); bitstream* stream_clone(const bitstream* stream); /* word size in bits (equal to stream_word_bits) */ -size_t stream_alignment(); +bitstream_count stream_alignment(); /* pointer to beginning of stream */ void* stream_data(const bitstream* stream); @@ -49,40 +58,40 @@ uint stream_read_bit(bitstream* stream); uint stream_write_bit(bitstream* stream, uint bit); /* read 0 <= n <= 64 bits */ -uint64 stream_read_bits(bitstream* stream, uint n); +uint64 stream_read_bits(bitstream* stream, bitstream_count n); /* write 0 <= n <= 64 low bits of value and return remaining bits */ -uint64 stream_write_bits(bitstream* stream, uint64 value, uint n); +uint64 stream_write_bits(bitstream* stream, uint64 value, bitstream_count n); /* return bit offset to next bit to be read */ -size_t stream_rtell(const bitstream* stream); +bitstream_offset stream_rtell(const bitstream* stream); /* return bit offset to next bit to be written */ -size_t stream_wtell(const bitstream* stream); +bitstream_offset stream_wtell(const bitstream* stream); /* rewind stream to beginning */ void stream_rewind(bitstream* stream); /* position stream for reading at given bit offset */ -void stream_rseek(bitstream* stream, size_t offset); +void stream_rseek(bitstream* stream, bitstream_offset offset); /* position stream for writing at given bit offset */ -void stream_wseek(bitstream* stream, size_t offset); +void stream_wseek(bitstream* stream, bitstream_offset offset); /* skip over the next n bits */ -void stream_skip(bitstream* stream, uint n); +void stream_skip(bitstream* stream, bitstream_size n); /* append n zero-bits to stream */ -void stream_pad(bitstream* stream, uint n); +void stream_pad(bitstream* stream, bitstream_size n); /* align stream on next word boundary */ -size_t stream_align(bitstream* stream); +bitstream_count stream_align(bitstream* stream); /* flush out any remaining buffered bits */ -size_t stream_flush(bitstream* stream); +bitstream_count stream_flush(bitstream* stream); /* copy n bits from one bit stream to another */ -void stream_copy(bitstream* dst, bitstream* src, size_t n); +void stream_copy(bitstream* dst, bitstream* src, bitstream_size n); #ifdef BIT_STREAM_STRIDED /* set block size in number of words and spacing in number of blocks */ diff --git a/src/cuda_zfp/cuZFP.cu b/src/cuda_zfp/cuZFP.cu index 657976e3d..619caca22 100644 --- a/src/cuda_zfp/cuZFP.cu +++ b/src/cuda_zfp/cuZFP.cu @@ -198,7 +198,7 @@ size_t decode(uint ndims[3], int3 stride, int bits_per_block, Word *stream, T *o Word *setup_device_stream_compress(zfp_stream *stream,const zfp_field *field) { bool stream_device = cuZFP::is_gpu_ptr(stream->stream->begin); - assert(sizeof(word) == sizeof(Word)); // "CUDA version currently only supports 64bit words"); + assert(sizeof(bitstream_word) == sizeof(Word)); // "CUDA version currently only supports 64bit words"); if(stream_device) { @@ -214,7 +214,7 @@ Word *setup_device_stream_compress(zfp_stream *stream,const zfp_field *field) Word *setup_device_stream_decompress(zfp_stream *stream,const zfp_field *field) { bool stream_device = cuZFP::is_gpu_ptr(stream->stream->begin); - assert(sizeof(word) == sizeof(Word)); // "CUDA version currently only supports 64bit words"); + assert(sizeof(bitstream_word) == sizeof(Word)); // "CUDA version currently only supports 64bit words"); if(stream_device) { diff --git a/src/inline/bitstream.c b/src/inline/bitstream.c index 3293ac343..a4a60f477 100644 --- a/src/inline/bitstream.c +++ b/src/inline/bitstream.c @@ -22,35 +22,36 @@ The following assumptions and restrictions apply: stream for writing. In read mode, the following functions may be called: size_t stream_size(stream); - size_t stream_rtell(stream); + bitstream_offset stream_rtell(stream); void stream_rewind(stream); void stream_rseek(stream, offset); - void stream_skip(stream, uint n); - size_t stream_align(stream); + void stream_skip(stream, n); + bitstream_count stream_align(stream); uint stream_read_bit(stream); uint64 stream_read_bits(stream, n); Each of the above read calls has a corresponding write call: size_t stream_size(stream); - size_t stream_wtell(stream); + bitstream_offset stream_wtell(stream); void stream_rewind(stream); void stream_wseek(stream, offset); void stream_pad(stream, n); - size_t stream_flush(stream); + bitstream_count stream_flush(stream); uint stream_write_bit(stream, bit); uint64 stream_write_bits(stream, value, n); 3. The stream buffer is an unsigned integer of a user-specified type given by the BIT_STREAM_WORD_TYPE macro. Bits are read and written in units of this integer word type. Supported types are 8, 16, 32, or 64 bits wide. - The bit width of the buffer is denoted by 'wsize' and can be accessed via - the global constant stream_word_bits. A small wsize allows for fine - granularity reads and writes, and may be preferable when working with many - small blocks of data that require non-sequential access. The default - maximum size of 64 bits ensures maximum speed. Note that even when - wsize < 64, it is still possible to read and write up to 64 bits at a time - using stream_read_bits() and stream_write_bits(). + The bit width of the buffer is denoted by 'wsize' and can be accessed + either via the global constant stream_word_bits or stream_alignment(). + A small wsize allows for fine granularity reads and writes, and may be + preferable when working with many small blocks of data that require + non-sequential access. The default maximum size of 64 bits ensures maximum + speed. Note that even when wsize < 64, it is still possible to read and + write up to 64 bits at a time using stream_read_bits() and + stream_write_bits(). 4. If BIT_STREAM_STRIDED is defined, words read from or written to the stream may be accessed noncontiguously by setting a power-of-two block size (which @@ -58,7 +59,7 @@ The following assumptions and restrictions apply: word pointer is always incremented by one word each time a word is accessed. Once advanced past a block boundary, the word pointer is also advanced by the stride to the next block. This feature may be used to store blocks of - data interleaved, e.g. for progressive coding or for noncontiguous parallel + data interleaved, e.g., for progressive coding or for noncontiguous parallel access to the bit stream Note that the block size is measured in words, while the stride is measured in multiples of the block size. Strided access can have a significant performance penalty. @@ -71,7 +72,7 @@ The following assumptions and restrictions apply: is essentially equivalent to (but faster than) for (i = 0; i < n; i++, value >>= 1) - stream_write_bit(value & 1); + stream_write_bit(stream, value & 1); when 0 <= n <= 64. The same holds for read calls, and thus @@ -80,11 +81,15 @@ The following assumptions and restrictions apply: is essentially equivalent to for (i = 0, value = 0; i < n; i++) - value += (uint64)stream_read_bit() << i; + value += (uint64)stream_read_bit(stream) << i; Note that it is possible to write fewer bits than the argument 'value' holds (possibly even no bits), in which case any unwritten bits are - returned. + shifted right to the least significant position and returned. That is, + value = stream_write_bits(stream, value, n); is equivalent to + + for (i = 0; i < n; i++) + value = stream_write_bits(stream, value, 1); 6. Although the stream_wseek(stream, offset) call allows positioning the stream for writing at any bit offset without any data loss (i.e. all @@ -107,41 +112,43 @@ The following assumptions and restrictions apply: #define inline_ #endif +#include "bitstream.h" + /* satisfy compiler when args unused */ #define unused_(x) ((void)(x)) /* bit stream word/buffer type; granularity of stream I/O operations */ #ifdef BIT_STREAM_WORD_TYPE /* may be 8-, 16-, 32-, or 64-bit unsigned integer type */ - typedef BIT_STREAM_WORD_TYPE stream_word; + typedef BIT_STREAM_WORD_TYPE bitstream_word; #else /* use maximum word size by default for highest speed */ - typedef uint64 stream_word; + typedef uint64 bitstream_word; #endif /* number of bits in a buffered word */ -#define wsize ((uint)(CHAR_BIT * sizeof(stream_word))) +#define wsize ((bitstream_count)(sizeof(bitstream_word) * CHAR_BIT)) /* bit stream structure (opaque to caller) */ struct bitstream { - uint bits; /* number of buffered bits (0 <= bits < wsize) */ - stream_word buffer; /* buffer for incoming/outgoing bits (buffer < 2^bits) */ - stream_word* ptr; /* pointer to next word to be read/written */ - stream_word* begin; /* beginning of stream */ - stream_word* end; /* end of stream (currently unused) */ + bitstream_count bits; /* number of buffered bits (0 <= bits < wsize) */ + bitstream_word buffer; /* incoming/outgoing bits (buffer < 2^bits) */ + bitstream_word* ptr; /* pointer to next word to be read/written */ + bitstream_word* begin; /* beginning of stream */ + bitstream_word* end; /* end of stream (not enforced) */ #ifdef BIT_STREAM_STRIDED - size_t mask; /* one less the block size in number of words */ - ptrdiff_t delta; /* number of words between consecutive blocks */ + size_t mask; /* one less the block size in number of words */ + ptrdiff_t delta; /* number of words between consecutive blocks */ #endif }; /* private functions ------------------------------------------------------- */ /* read a single word from memory */ -static stream_word +static bitstream_word stream_read_word(bitstream* s) { - stream_word w = *s->ptr++; + bitstream_word w = *s->ptr++; #ifdef BIT_STREAM_STRIDED if (!((s->ptr - s->begin) & s->mask)) s->ptr += s->delta; @@ -151,7 +158,7 @@ stream_read_word(bitstream* s) /* write a single word to memory */ static void -stream_write_word(bitstream* s, stream_word value) +stream_write_word(bitstream* s, bitstream_word value) { *s->ptr++ = value; #ifdef BIT_STREAM_STRIDED @@ -162,8 +169,8 @@ stream_write_word(bitstream* s, stream_word value) /* public functions -------------------------------------------------------- */ -/* word size in bits (equals stream_word_bits) */ -inline_ size_t +/* word size in bits (equals bitstream_word_bits) */ +inline_ bitstream_count stream_alignment() { return wsize; @@ -180,14 +187,14 @@ stream_data(const bitstream* s) inline_ size_t stream_size(const bitstream* s) { - return sizeof(stream_word) * (size_t)(s->ptr - s->begin); + return (size_t)(s->ptr - s->begin) * sizeof(bitstream_word); } /* byte capacity of stream */ inline_ size_t stream_capacity(const bitstream* s) { - return sizeof(stream_word) * (size_t)(s->end - s->begin); + return (size_t)(s->end - s->begin) * sizeof(bitstream_word); } /* number of words per block */ @@ -233,7 +240,7 @@ stream_read_bit(bitstream* s) inline_ uint stream_write_bit(bitstream* s, uint bit) { - s->buffer += (stream_word)bit << s->bits; + s->buffer += (bitstream_word)bit << s->bits; if (++s->bits == wsize) { stream_write_word(s, s->buffer); s->buffer = 0; @@ -244,7 +251,7 @@ stream_write_bit(bitstream* s, uint bit) /* read 0 <= n <= 64 bits */ inline_ uint64 -stream_read_bits(bitstream* s, uint n) +stream_read_bits(bitstream* s, bitstream_count n) { uint64 value = s->buffer; if (s->bits < n) { @@ -279,10 +286,10 @@ stream_read_bits(bitstream* s, uint n) /* write 0 <= n <= 64 low bits of value and return remaining bits */ inline_ uint64 -stream_write_bits(bitstream* s, uint64 value, uint n) +stream_write_bits(bitstream* s, uint64 value, bitstream_count n) { /* append bit string to buffer */ - s->buffer += (stream_word)(value << s->bits); + s->buffer += (bitstream_word)(value << s->bits); s->bits += n; /* is buffer full? */ if (s->bits >= wsize) { @@ -296,27 +303,27 @@ stream_write_bits(bitstream* s, uint64 value, uint n) /* assert: 0 <= s->bits <= n */ stream_write_word(s, s->buffer); /* assert: 0 <= n - s->bits < 64 */ - s->buffer = (stream_word)(value >> (n - s->bits)); + s->buffer = (bitstream_word)(value >> (n - s->bits)); } while (sizeof(s->buffer) < sizeof(value) && s->bits >= wsize); } /* assert: 0 <= s->bits < wsize */ - s->buffer &= ((stream_word)1 << s->bits) - 1; + s->buffer &= ((bitstream_word)1 << s->bits) - 1; /* assert: 0 <= n < 64 */ return value >> n; } /* return bit offset to next bit to be read */ -inline_ size_t +inline_ bitstream_offset stream_rtell(const bitstream* s) { - return wsize * (size_t)(s->ptr - s->begin) - s->bits; + return (bitstream_offset)(s->ptr - s->begin) * wsize - s->bits; } /* return bit offset to next bit to be written */ -inline_ size_t +inline_ bitstream_offset stream_wtell(const bitstream* s) { - return wsize * (size_t)(s->ptr - s->begin) + s->bits; + return (bitstream_offset)(s->ptr - s->begin) * wsize + s->bits; } /* position stream for reading or writing at beginning */ @@ -330,10 +337,10 @@ stream_rewind(bitstream* s) /* position stream for reading at given bit offset */ inline_ void -stream_rseek(bitstream* s, size_t offset) +stream_rseek(bitstream* s, bitstream_offset offset) { - uint n = offset % wsize; - s->ptr = s->begin + offset / wsize; + bitstream_count n = (bitstream_count)(offset % wsize); + s->ptr = s->begin + (size_t)(offset / wsize); if (n) { s->buffer = stream_read_word(s) >> n; s->bits = wsize - n; @@ -346,13 +353,13 @@ stream_rseek(bitstream* s, size_t offset) /* position stream for writing at given bit offset */ inline_ void -stream_wseek(bitstream* s, size_t offset) +stream_wseek(bitstream* s, bitstream_offset offset) { - uint n = offset % wsize; - s->ptr = s->begin + offset / wsize; + bitstream_count n = (bitstream_count)(offset % wsize); + s->ptr = s->begin + (size_t)(offset / wsize); if (n) { - stream_word buffer = *s->ptr; - buffer &= ((stream_word)1 << n) - 1; + bitstream_word buffer = *s->ptr; + buffer &= ((bitstream_word)1 << n) - 1; s->buffer = buffer; s->bits = n; } @@ -364,36 +371,38 @@ stream_wseek(bitstream* s, size_t offset) /* skip over the next n bits (n >= 0) */ inline_ void -stream_skip(bitstream* s, uint n) +stream_skip(bitstream* s, bitstream_size n) { stream_rseek(s, stream_rtell(s) + n); } /* append n zero-bits to stream (n >= 0) */ inline_ void -stream_pad(bitstream* s, uint n) +stream_pad(bitstream* s, bitstream_size n) { - for (s->bits += n; s->bits >= wsize; s->bits -= wsize) { + bitstream_offset bits = s->bits; + for (bits += n; bits >= wsize; bits -= wsize) { stream_write_word(s, s->buffer); s->buffer = 0; } + s->bits = (bitstream_count)bits; } /* align stream on next word boundary */ -inline_ size_t +inline_ bitstream_count stream_align(bitstream* s) { - uint bits = s->bits; + bitstream_count bits = s->bits; if (bits) stream_skip(s, bits); return bits; } /* write any remaining buffered bits and align stream on next word boundary */ -inline_ size_t +inline_ bitstream_count stream_flush(bitstream* s) { - uint bits = (wsize - s->bits) % wsize; + bitstream_count bits = (wsize - s->bits) % wsize; if (bits) stream_pad(s, bits); return bits; @@ -401,16 +410,16 @@ stream_flush(bitstream* s) /* copy n bits from one bit stream to another */ inline_ void -stream_copy(bitstream* dst, bitstream* src, size_t n) +stream_copy(bitstream* dst, bitstream* src, bitstream_size n) { while (n > wsize) { - stream_word w = (stream_word)stream_read_bits(src, wsize); + bitstream_word w = (bitstream_word)stream_read_bits(src, wsize); stream_write_bits(dst, w, wsize); n -= wsize; } if (n) { - stream_word w = (stream_word)stream_read_bits(src, (uint)n); - stream_write_bits(dst, w, (uint)n); + bitstream_word w = (bitstream_word)stream_read_bits(src, (bitstream_count)n); + stream_write_bits(dst, w, (bitstream_count)n); } } @@ -434,8 +443,8 @@ stream_open(void* buffer, size_t bytes) { bitstream* s = (bitstream*)malloc(sizeof(bitstream)); if (s) { - s->begin = (stream_word*)buffer; - s->end = s->begin + bytes / sizeof(stream_word); + s->begin = (bitstream_word*)buffer; + s->end = s->begin + bytes / sizeof(bitstream_word); #ifdef BIT_STREAM_STRIDED stream_set_stride(s, 0, 0); #endif diff --git a/src/share/parallel.c b/src/share/parallel.c index 0cbdc02c3..1ae365268 100644 --- a/src/share/parallel.c +++ b/src/share/parallel.c @@ -79,12 +79,12 @@ compress_finish_par(zfp_stream* stream, bitstream** src, size_t chunks) { bitstream* dst = zfp_stream_bit_stream(stream); zfp_bool copy = (stream_data(dst) != stream_data(*src)); - size_t offset = stream_wtell(dst); + bitstream_offset offset = stream_wtell(dst); size_t chunk; /* flush each stream and concatenate if necessary */ for (chunk = 0; chunk < chunks; chunk++) { - size_t bits = stream_wtell(src[chunk]); + bitstream_size bits = stream_wtell(src[chunk]); offset += bits; stream_flush(src[chunk]); /* concatenate streams if they are not already contiguous */ diff --git a/src/template/decode.c b/src/template/decode.c index 990f83860..2df1a958a 100644 --- a/src/template/decode.c +++ b/src/template/decode.c @@ -178,7 +178,7 @@ _t1(decode_few_ints_prec, UInt)(bitstream* restrict_ stream, uint maxprec, UInt* { /* make a copy of bit stream to avoid aliasing */ bitstream s = *stream; - size_t offset = stream_rtell(&s); + bitstream_offset offset = stream_rtell(&s); uint intprec = (uint)(CHAR_BIT * sizeof(UInt)); uint kmin = intprec > maxprec ? intprec - maxprec : 0; uint i, k, n; @@ -215,7 +215,7 @@ _t1(decode_many_ints_prec, UInt)(bitstream* restrict_ stream, uint maxprec, UInt { /* make a copy of bit stream to avoid aliasing */ bitstream s = *stream; - size_t offset = stream_rtell(&s); + bitstream_offset offset = stream_rtell(&s); uint intprec = (uint)(CHAR_BIT * sizeof(UInt)); uint kmin = intprec > maxprec ? intprec - maxprec : 0; uint i, k, n; diff --git a/src/template/encode.c b/src/template/encode.c index e2f519b7e..0d63bb1f3 100644 --- a/src/template/encode.c +++ b/src/template/encode.c @@ -181,7 +181,7 @@ _t1(encode_few_ints_prec, UInt)(bitstream* restrict_ stream, uint maxprec, const { /* make a copy of bit stream to avoid aliasing */ bitstream s = *stream; - size_t offset = stream_wtell(&s); + bitstream_offset offset = stream_wtell(&s); uint intprec = (uint)(CHAR_BIT * sizeof(UInt)); uint kmin = intprec > maxprec ? intprec - maxprec : 0; uint i, k, n; @@ -210,7 +210,7 @@ _t1(encode_many_ints_prec, UInt)(bitstream* restrict_ stream, uint maxprec, cons { /* make a copy of bit stream to avoid aliasing */ bitstream s = *stream; - size_t offset = stream_wtell(&s); + bitstream_offset offset = stream_wtell(&s); uint intprec = (uint)(CHAR_BIT * sizeof(UInt)); uint kmin = intprec > maxprec ? intprec - maxprec : 0; uint i, k, n, c; diff --git a/tests/src/inline/testBitstream.c b/tests/src/inline/testBitstream.c index 948a77259..904470b0e 100644 --- a/tests/src/inline/testBitstream.c +++ b/tests/src/inline/testBitstream.c @@ -8,7 +8,7 @@ #define STREAM_WORD_CAPACITY 3 -#define WORD_MASK ((stream_word)(-1)) +#define WORD_MASK ((bitstream_word)(-1)) #define WORD1 WORD_MASK #define WORD2 (0x5555555555555555 & WORD_MASK) @@ -23,10 +23,10 @@ setup(void **state) struct setupVars *s = malloc(sizeof(struct setupVars)); assert_non_null(s); - s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(stream_word)); + s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(bitstream_word)); assert_non_null(s->buffer); - s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(stream_word)); + s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(bitstream_word)); assert_non_null(s->b); *state = s; @@ -53,11 +53,11 @@ when_StreamCopy_expect_BitsCopiedToDestBitstream(void **state) const uint COPY_BITS = wsize + 4; const uint NUM_WORD2_BITS_WRITTEN_TO_WORD = DST_OFFSET + (wsize - SRC_OFFSET); - const stream_word EXPECTED_WRITTEN_WORD = ((WORD1 >> SRC_OFFSET) << DST_OFFSET) - + (WORD2 << NUM_WORD2_BITS_WRITTEN_TO_WORD); + const bitstream_word EXPECTED_WRITTEN_WORD = ((WORD1 >> SRC_OFFSET) << DST_OFFSET) + + (WORD2 << NUM_WORD2_BITS_WRITTEN_TO_WORD); const uint EXPECTED_BITS = (DST_OFFSET + COPY_BITS) % wsize; - const stream_word EXPECTED_BUFFER = (WORD2 >> (NUM_WORD2_BITS_WRITTEN_TO_WORD)) - & ((1u << EXPECTED_BITS) - 1); + const bitstream_word EXPECTED_BUFFER = (WORD2 >> (NUM_WORD2_BITS_WRITTEN_TO_WORD)) + & ((1u << EXPECTED_BITS) - 1); bitstream* src = ((struct setupVars *)*state)->b; stream_write_word(src, WORD1); @@ -65,8 +65,8 @@ when_StreamCopy_expect_BitsCopiedToDestBitstream(void **state) stream_flush(src); stream_rseek(src, SRC_OFFSET); - void* buffer = calloc(STREAM_WORD_CAPACITY, sizeof(stream_word)); - bitstream* dst = stream_open(buffer, STREAM_WORD_CAPACITY * sizeof(stream_word)); + void* buffer = calloc(STREAM_WORD_CAPACITY, sizeof(bitstream_word)); + bitstream* dst = stream_open(buffer, STREAM_WORD_CAPACITY * sizeof(bitstream_word)); stream_wseek(dst, DST_OFFSET); stream_copy(dst, src, COPY_BITS); @@ -91,7 +91,7 @@ when_Flush_expect_PaddedWordWrittenToStream(void **state) stream_rewind(s); stream_write_bits(s, WORD2, PREV_BUFFER_BIT_COUNT); - stream_word *prevPtr = s->ptr; + bitstream_word *prevPtr = s->ptr; size_t padCount = stream_flush(s); @@ -105,9 +105,9 @@ static void given_EmptyBuffer_when_Flush_expect_NOP(void **state) { bitstream* s = ((struct setupVars *)*state)->b; - stream_word *prevPtr = s->ptr; + bitstream_word *prevPtr = s->ptr; uint prevBits = s->bits; - stream_word prevBuffer = s->buffer; + bitstream_word prevBuffer = s->buffer; size_t padCount = stream_flush(s); @@ -128,7 +128,7 @@ when_Align_expect_BufferEmptyBitsZero(void **state) stream_rewind(s); stream_read_bits(s, READ_BIT_COUNT); - stream_word *prevPtr = s->ptr; + bitstream_word *prevPtr = s->ptr; stream_align(s); @@ -144,7 +144,7 @@ when_SkipPastBufferEnd_expect_NewMaskedWordInBuffer(void **state) const uint SKIP_COUNT = wsize + 5; const uint TOTAL_OFFSET = READ_BIT_COUNT + SKIP_COUNT; const uint EXPECTED_BITS = wsize - (TOTAL_OFFSET % wsize); - const stream_word EXPECTED_BUFFER = WORD2 >> (TOTAL_OFFSET % wsize); + const bitstream_word EXPECTED_BUFFER = WORD2 >> (TOTAL_OFFSET % wsize); bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); @@ -167,14 +167,14 @@ when_SkipWithinBuffer_expect_MaskedBuffer(void **state) const uint SKIP_COUNT = 5; const uint TOTAL_OFFSET = READ_BIT_COUNT + SKIP_COUNT; const uint EXPECTED_BITS = wsize - (TOTAL_OFFSET % wsize); - const stream_word EXPECTED_BUFFER = WORD1 >> (TOTAL_OFFSET % wsize); + const bitstream_word EXPECTED_BUFFER = WORD1 >> (TOTAL_OFFSET % wsize); bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); stream_rewind(s); stream_read_bits(s, READ_BIT_COUNT); - stream_word *prevPtr = s->ptr; + bitstream_word *prevPtr = s->ptr; stream_skip(s, SKIP_COUNT); @@ -193,9 +193,9 @@ when_SkipZeroBits_expect_NOP(void **state) stream_rewind(s); stream_read_bits(s, 2); - stream_word* prevPtr = s->ptr; - stream_word prevBits = s->bits; - stream_word prevBuffer = s->buffer; + bitstream_word* prevPtr = s->ptr; + bitstream_word prevBits = s->bits; + bitstream_word prevBuffer = s->buffer; stream_skip(s, 0); @@ -209,7 +209,7 @@ when_RseekToNonMultipleOfWsize_expect_MaskedWordLoadedToBuffer(void **state) { const uint BIT_OFFSET = wsize + 5; const uint EXPECTED_BITS = wsize - (BIT_OFFSET % wsize); - const stream_word EXPECTED_BUFFER = WORD2 >> (BIT_OFFSET % wsize); + const bitstream_word EXPECTED_BUFFER = WORD2 >> (BIT_OFFSET % wsize); bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); @@ -240,7 +240,7 @@ static void when_WseekToNonMultipleOfWsize_expect_MaskedWordLoadedToBuffer(void **state) { const uint BIT_OFFSET = wsize + 5; - const stream_word MASK = 0x1f; + const bitstream_word MASK = 0x1f; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, wsize); @@ -305,8 +305,8 @@ when_ReadBitsSpreadsAcrossTwoWords_expect_BitsCombinedFromBothWords(void **state const uint NUM_OVERFLOWED_BITS = READ_BIT_COUNT - PARTIAL_WORD_BIT_COUNT; const uint EXPECTED_BUFFER_BIT_COUNT = wsize - NUM_OVERFLOWED_BITS; - const stream_word PARTIAL_WORD1 = WORD1 & 0xffff; - const stream_word PARTIAL_WORD2 = WORD2 & 0x1fffffffffff << PARTIAL_WORD_BIT_COUNT; + const bitstream_word PARTIAL_WORD1 = WORD1 & 0xffff; + const bitstream_word PARTIAL_WORD2 = WORD2 & 0x1fffffffffff << PARTIAL_WORD_BIT_COUNT; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, PARTIAL_WORD1, wsize); @@ -347,7 +347,7 @@ static void when_ReadBits_expect_BitsReadInOrderLSB(void **state) { const uint BITS_TO_READ = 2; - const stream_word MASK = 0x3; + const bitstream_word MASK = 0x3; bitstream* s = ((struct setupVars *)*state)->b; s->buffer = WORD2; @@ -383,9 +383,9 @@ when_WriteBitsOverflowsBuffer_expect_OverflowWrittenToNewBuffer(void **state) const uint NUM_BITS_TO_WRITE = wsize - 1; const uint OVERFLOW_BIT_COUNT = NUM_BITS_TO_WRITE - (wsize - EXISTING_BIT_COUNT); // 0x1101 0101 0101 ... 0101 allows stream_write_bit() to return non-zero - const stream_word WORD_TO_WRITE = WORD2 + 0x8000000000000000; - const stream_word OVERFLOWED_BITS = WORD_TO_WRITE >> (wsize - EXISTING_BIT_COUNT); - const stream_word EXPECTED_BUFFER_RESULT = OVERFLOWED_BITS & 0xf; + const bitstream_word WORD_TO_WRITE = WORD2 + 0x8000000000000000; + const bitstream_word OVERFLOWED_BITS = WORD_TO_WRITE >> (wsize - EXISTING_BIT_COUNT); + const bitstream_word EXPECTED_BUFFER_RESULT = OVERFLOWED_BITS & 0xf; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, EXISTING_BIT_COUNT); @@ -402,14 +402,14 @@ when_WriteBitsFillsBufferExactly_expect_WordWrittenToStream(void **state) { const uint EXISTING_BIT_COUNT = 5; const uint NUM_BITS_TO_WRITE = wsize - EXISTING_BIT_COUNT; - const stream_word COMPLETING_WORD = WORD2 & 0x07ffffffffffffff; + const bitstream_word COMPLETING_WORD = WORD2 & 0x07ffffffffffffff; bitstream* s = ((struct setupVars *)*state)->b; stream_write_bits(s, WORD1, EXISTING_BIT_COUNT); uint64 remainingBits = stream_write_bits(s, COMPLETING_WORD, NUM_BITS_TO_WRITE); stream_rewind(s); - stream_word readWord = stream_read_word(s); + bitstream_word readWord = stream_read_word(s); assert_int_equal(readWord, 0x1f + 0xaaaaaaaaaaaaaaa0); assert_int_equal(remainingBits, 0); @@ -465,7 +465,7 @@ given_BitstreamWithBitInBuffer_when_ReadBit_expect_OneBitReadFromLSB(void **stat stream_write_bit(s, 1); uint prevBits = s->bits; - stream_word prevBuffer = s->buffer; + bitstream_word prevBuffer = s->buffer; assert_int_equal(stream_read_bit(s), 1); assert_int_equal(s->bits, prevBits - 1); @@ -482,8 +482,8 @@ given_BitstreamBufferOneBitFromFull_when_WriteBit_expect_BitWrittenToBufferWritt stream_write_bit(s, 1); - assert_int_equal(stream_size(s), sizeof(stream_word)); - assert_int_equal(*s->begin, (stream_word)1 << PLACE); + assert_int_equal(stream_size(s), sizeof(bitstream_word)); + assert_int_equal(*s->begin, (bitstream_word)1 << PLACE); assert_int_equal(s->buffer, 0); } @@ -498,7 +498,7 @@ when_WriteBit_expect_BitWrittenToBufferFromLSB(void **state) stream_write_bit(s, 1); assert_int_equal(s->bits, PLACE + 1); - assert_int_equal(s->buffer, (stream_word)1 << PLACE); + assert_int_equal(s->buffer, (bitstream_word)1 << PLACE); } static void @@ -506,7 +506,7 @@ given_StartedBuffer_when_StreamPadOverflowsBuffer_expect_ProperWordsWritten(void { const uint NUM_WORDS = 2; const uint EXISTING_BIT_COUNT = 12; - const stream_word EXISTING_BUFFER = 0xfff; + const bitstream_word EXISTING_BUFFER = 0xfff; const uint PAD_AMOUNT = NUM_WORDS * wsize - EXISTING_BIT_COUNT; bitstream* s = ((struct setupVars *)*state)->b; @@ -520,7 +520,7 @@ given_StartedBuffer_when_StreamPadOverflowsBuffer_expect_ProperWordsWritten(void stream_pad(s, PAD_AMOUNT); - assert_int_equal(stream_size(s), prevStreamSize + NUM_WORDS * sizeof(stream_word)); + assert_int_equal(stream_size(s), prevStreamSize + NUM_WORDS * sizeof(bitstream_word)); stream_rewind(s); assert_int_equal(stream_read_word(s), EXISTING_BUFFER); assert_int_equal(stream_read_word(s), 0); @@ -530,7 +530,7 @@ static void given_StartedBuffer_when_StreamPad_expect_PaddedWordWritten(void **state) { const uint EXISTING_BIT_COUNT = 12; - const stream_word EXISTING_BUFFER = 0xfff; + const bitstream_word EXISTING_BUFFER = 0xfff; bitstream* s = ((struct setupVars *)*state)->b; s->buffer = EXISTING_BUFFER; @@ -539,7 +539,7 @@ given_StartedBuffer_when_StreamPad_expect_PaddedWordWritten(void **state) stream_pad(s, wsize - EXISTING_BIT_COUNT); - assert_int_equal(stream_size(s), prevStreamSize + sizeof(stream_word)); + assert_int_equal(stream_size(s), prevStreamSize + sizeof(bitstream_word)); stream_rewind(s); assert_int_equal(stream_read_word(s), EXISTING_BUFFER); } @@ -586,7 +586,7 @@ when_WriteTwoWords_expect_WordsWrittenToStreamConsecutively(void **state) stream_write_word(s, WORD1); stream_write_word(s, WORD2); - assert_int_equal(stream_size(s), sizeof(stream_word) * 2); + assert_int_equal(stream_size(s), sizeof(bitstream_word) * 2); assert_int_equal(*s->begin, WORD1); assert_int_equal(*(s->begin + 1), WORD2); } @@ -599,7 +599,7 @@ given_RewoundBitstream_when_WriteWord_expect_WordWrittenAtStreamBegin(void **sta stream_write_word(s, WORD1); - assert_int_equal(stream_size(s), prevStreamSize + sizeof(stream_word)); + assert_int_equal(stream_size(s), prevStreamSize + sizeof(bitstream_word)); assert_int_equal(*s->begin, WORD1); } @@ -608,12 +608,12 @@ when_BitstreamOpened_expect_ProperLengthAndBoundaries(void **state) { const int NUM_WORDS = 4; - size_t bufferLenBytes = sizeof(stream_word) * NUM_WORDS; + size_t bufferLenBytes = sizeof(bitstream_word) * NUM_WORDS; void* buffer = malloc(bufferLenBytes); bitstream* s = stream_open(buffer, bufferLenBytes); void* streamBegin = stream_data(s); - void* computedStreamEnd = (stream_word*)streamBegin + NUM_WORDS; + void* computedStreamEnd = (bitstream_word*)streamBegin + NUM_WORDS; assert_ptr_equal(streamBegin, buffer); assert_ptr_equal(s->end, computedStreamEnd); diff --git a/tests/src/inline/testBitstreamSmallWsize.c b/tests/src/inline/testBitstreamSmallWsize.c index 4bed088e3..1c4f96295 100644 --- a/tests/src/inline/testBitstreamSmallWsize.c +++ b/tests/src/inline/testBitstreamSmallWsize.c @@ -21,10 +21,10 @@ setup(void **state) struct setupVars *s = malloc(sizeof(struct setupVars)); assert_non_null(s); - s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(stream_word)); + s->buffer = calloc(STREAM_WORD_CAPACITY, sizeof(bitstream_word)); assert_non_null(s->buffer); - s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(stream_word)); + s->b = stream_open(s->buffer, STREAM_WORD_CAPACITY * sizeof(bitstream_word)); assert_non_null(s->b); *state = s; @@ -72,7 +72,7 @@ when_ReadBitsSpreadsAcrossMultipleWords_expect_BitsCombinedFromMultipleWords(voi + (WRITE_BITS2 << PARTIAL_WORD_BIT_COUNT) + (WRITE_BITS3 << (wsize + PARTIAL_WORD_BIT_COUNT)) + ((WRITE_BITS4 & 0xff) << (2*wsize + PARTIAL_WORD_BIT_COUNT))); - assert_int_equal(s->buffer, (stream_word) (WRITE_BITS4 >> (NUM_OVERFLOWED_BITS % wsize))); + assert_int_equal(s->buffer, (bitstream_word) (WRITE_BITS4 >> (NUM_OVERFLOWED_BITS % wsize))); } // overflow refers to what will land in the buffer diff --git a/tests/src/inline/testBitstreamStrided.c b/tests/src/inline/testBitstreamStrided.c index 649211b6b..a7fa2c7a3 100644 --- a/tests/src/inline/testBitstreamStrided.c +++ b/tests/src/inline/testBitstreamStrided.c @@ -26,10 +26,10 @@ setup(void **state) struct setupVars *s = malloc(sizeof(struct setupVars)); assert_non_null(s); - s->buffer = calloc(STREAM_STRIDED_LEN, sizeof(stream_word)); + s->buffer = calloc(STREAM_STRIDED_LEN, sizeof(bitstream_word)); assert_non_null(s->buffer); - s->b = stream_open(s->buffer, STREAM_STRIDED_LEN * sizeof(stream_word)); + s->b = stream_open(s->buffer, STREAM_STRIDED_LEN * sizeof(bitstream_word)); assert_non_null(s->b); assert_true(stream_set_stride(s->b, BLOCK_SIZE, DELTA)); @@ -54,7 +54,7 @@ static void given_Strided_when_ReadWordCompletesBlock_expect_PtrAdvancedByStrideLen(void **state) { bitstream* s = ((struct setupVars *)*state)->b; - stream_word* prevPtr = s->ptr; + bitstream_word* prevPtr = s->ptr; int i; for (i = 0; i < BLOCK_SIZE - 1; i++) { @@ -71,7 +71,7 @@ static void given_Strided_when_WriteWordCompletesBlock_expect_PtrAdvancedByStrideLen(void **state) { bitstream* s = ((struct setupVars *)*state)->b; - stream_word* prevPtr = s->ptr; + bitstream_word* prevPtr = s->ptr; int i; for (i = 0; i < BLOCK_SIZE - 1; i++) { From bf718a6042fb194235dcd7d7dcf27575a65fcada Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 16 Jun 2022 10:29:17 -0700 Subject: [PATCH 058/277] Update bitstream types in CUDA tests --- tests/src/endtoend/cudaExecBase.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/src/endtoend/cudaExecBase.c b/tests/src/endtoend/cudaExecBase.c index 0e4042027..6e6750324 100644 --- a/tests/src/endtoend/cudaExecBase.c +++ b/tests/src/endtoend/cudaExecBase.c @@ -42,9 +42,9 @@ runZfpCompressDecompressIsNoop(void **state) bitstream* s = zfp_stream_bit_stream(stream); // grab bitstream member vars - uint bits = s->bits; - word buffer = s->buffer; - word* ptr = s->ptr; + bitstream_count bits = s->bits; + bitstream_word buffer = s->buffer; + bitstream_word* ptr = s->ptr; size_t streamSize = stream_size(s); // perform compression, expect bitstream not to advance From 2d975ce8942bcaeead9770520416224e633f4679 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 16 Jun 2022 13:00:03 -0700 Subject: [PATCH 059/277] Correct type mismatches to silence compiler warnings --- array/zfp/store.h | 2 +- tests/src/inline/testBitstream.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/array/zfp/store.h b/array/zfp/store.h index 554638051..7f91c38fe 100644 --- a/array/zfp/store.h +++ b/array/zfp/store.h @@ -203,7 +203,7 @@ class BlockStore { } // bit offset to block store - size_t offset(size_t block_index) const { return index.block_offset(block_index); } + bitstream_offset offset(size_t block_index) const { return index.block_offset(block_index); } // shape 0 <= m <= 3 of block containing index i, 0 <= i <= n - 1 static uint shape_code(size_t i, size_t n) diff --git a/tests/src/inline/testBitstream.c b/tests/src/inline/testBitstream.c index 904470b0e..a8ac4cd76 100644 --- a/tests/src/inline/testBitstream.c +++ b/tests/src/inline/testBitstream.c @@ -106,7 +106,7 @@ given_EmptyBuffer_when_Flush_expect_NOP(void **state) { bitstream* s = ((struct setupVars *)*state)->b; bitstream_word *prevPtr = s->ptr; - uint prevBits = s->bits; + bitstream_count prevBits = s->bits; bitstream_word prevBuffer = s->buffer; size_t padCount = stream_flush(s); @@ -464,7 +464,7 @@ given_BitstreamWithBitInBuffer_when_ReadBit_expect_OneBitReadFromLSB(void **stat bitstream* s = ((struct setupVars *)*state)->b; stream_write_bit(s, 1); - uint prevBits = s->bits; + bitstream_count prevBits = s->bits; bitstream_word prevBuffer = s->buffer; assert_int_equal(stream_read_bit(s), 1); From 938ffd2e95aebd3d5ee2dffd7cef8b4f5cea454d Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 22 Jun 2022 13:53:26 -0700 Subject: [PATCH 060/277] change exec policy params to use void* --- include/zfp.h | 7 +------ src/share/omp.c | 5 +++-- src/zfp.c | 29 +++++++++++++++++++++++------ 3 files changed, 27 insertions(+), 14 deletions(-) diff --git a/include/zfp.h b/include/zfp.h index 9cd5f5bbd..089ab94b6 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -77,14 +77,9 @@ typedef struct { uint chunk_size; /* number of blocks per chunk (1D only) */ } zfp_exec_params_omp; -/* execution parameters */ -typedef union { - zfp_exec_params_omp omp; /* OpenMP parameters */ -} zfp_exec_params; - typedef struct { zfp_exec_policy policy; /* execution policy (serial, omp, ...) */ - zfp_exec_params params; /* execution parameters */ + void* params; /* execution parameters */ } zfp_execution; /* compressed stream; use accessors to get/set members */ diff --git a/src/share/omp.c b/src/share/omp.c index e0300cadc..02507e564 100644 --- a/src/share/omp.c +++ b/src/share/omp.c @@ -1,12 +1,13 @@ #ifdef _OPENMP #include #include +#include "zfp.h" /* number of omp threads to use */ static uint thread_count_omp(const zfp_stream* stream) { - uint count = stream->exec.params.omp.threads; + uint count = zfp_stream_omp_threads(stream); /* if no thread count is specified, use default number of threads */ if (!count) count = omp_get_max_threads(); @@ -17,7 +18,7 @@ thread_count_omp(const zfp_stream* stream) static size_t chunk_count_omp(const zfp_stream* stream, size_t blocks, uint threads) { - size_t chunk_size = stream->exec.params.omp.chunk_size; + size_t chunk_size = (size_t)zfp_stream_omp_chunk_size(stream); /* if no chunk size is specified, assign one chunk per thread */ size_t chunks = chunk_size ? (blocks + chunk_size - 1) / chunk_size : threads; /* each chunk must contain at least one block */ diff --git a/src/zfp.c b/src/zfp.c index 179193dd8..a9c1b3381 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -528,6 +528,7 @@ zfp_stream_open(bitstream* stream) zfp->maxprec = ZFP_MAX_PREC; zfp->minexp = ZFP_MIN_EXP; zfp->exec.policy = zfp_exec_serial; + zfp->exec.params = NULL; } return zfp; } @@ -535,6 +536,8 @@ zfp_stream_open(bitstream* stream) void zfp_stream_close(zfp_stream* zfp) { + if (zfp->exec.params != NULL) + free(zfp->exec.params); free(zfp); } @@ -886,13 +889,13 @@ zfp_stream_execution(const zfp_stream* zfp) uint zfp_stream_omp_threads(const zfp_stream* zfp) { - return zfp->exec.params.omp.threads; + return ((zfp_exec_params_omp*)zfp->exec.params)->threads; } uint zfp_stream_omp_chunk_size(const zfp_stream* zfp) { - return zfp->exec.params.omp.chunk_size; + return ((zfp_exec_params_omp*)zfp->exec.params)->chunk_size; } zfp_bool @@ -900,16 +903,30 @@ zfp_stream_set_execution(zfp_stream* zfp, zfp_exec_policy policy) { switch (policy) { case zfp_exec_serial: + if (zfp->exec.policy != policy && zfp->exec.params != NULL) { + free(zfp->exec.params); + zfp->exec.params = NULL; + } break; #ifdef ZFP_WITH_CUDA case zfp_exec_cuda: + if (zfp->exec.policy != policy && zfp->exec.params != NULL) { + free(zfp->exec.params); + zfp->exec.params = NULL; + } break; #endif case zfp_exec_omp: #ifdef _OPENMP if (zfp->exec.policy != policy) { - zfp->exec.params.omp.threads = 0; - zfp->exec.params.omp.chunk_size = 0; + if (zfp->exec.params != NULL) { + free(zfp->exec.params); + zfp->exec.params = NULL; + } + zfp_exec_params_omp* params = malloc(sizeof(zfp_exec_params_omp)); + params->threads = 0; + params->chunk_size = 0; + zfp->exec.params = (void*)params; } break; #else @@ -927,7 +944,7 @@ zfp_stream_set_omp_threads(zfp_stream* zfp, uint threads) { if (!zfp_stream_set_execution(zfp, zfp_exec_omp)) return zfp_false; - zfp->exec.params.omp.threads = threads; + ((zfp_exec_params_omp*)zfp->exec.params)->threads = threads; return zfp_true; } @@ -936,7 +953,7 @@ zfp_stream_set_omp_chunk_size(zfp_stream* zfp, uint chunk_size) { if (!zfp_stream_set_execution(zfp, zfp_exec_omp)) return zfp_false; - zfp->exec.params.omp.chunk_size = chunk_size; + ((zfp_exec_params_omp*)zfp->exec.params)->chunk_size = chunk_size; return zfp_true; } From cc2e85008cd9fd859d7805ed952e57dcf1d9378d Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 23 Jun 2022 10:48:33 -0700 Subject: [PATCH 061/277] add policy mode validation checks --- src/zfp.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/zfp.c b/src/zfp.c index a9c1b3381..81625ad7b 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -889,13 +889,17 @@ zfp_stream_execution(const zfp_stream* zfp) uint zfp_stream_omp_threads(const zfp_stream* zfp) { - return ((zfp_exec_params_omp*)zfp->exec.params)->threads; + if (zfp->exec.policy == zfp_exec_omp) + return ((zfp_exec_params_omp*)zfp->exec.params)->threads; + return 0u; } uint zfp_stream_omp_chunk_size(const zfp_stream* zfp) { - return ((zfp_exec_params_omp*)zfp->exec.params)->chunk_size; + if (zfp->exec.policy == zfp_exec_omp) + return ((zfp_exec_params_omp*)zfp->exec.params)->chunk_size; + return 0u; } zfp_bool @@ -921,7 +925,6 @@ zfp_stream_set_execution(zfp_stream* zfp, zfp_exec_policy policy) if (zfp->exec.policy != policy) { if (zfp->exec.params != NULL) { free(zfp->exec.params); - zfp->exec.params = NULL; } zfp_exec_params_omp* params = malloc(sizeof(zfp_exec_params_omp)); params->threads = 0; From 537c1205261eaa982dc3a387f7e2a2fd8eead715 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 29 Jun 2022 18:05:27 -0700 Subject: [PATCH 062/277] Add compressed-array example --- README.md | 3 +++ docs/source/defs.rst | 1 + docs/source/examples.rst | 17 ++++++++++++-- docs/source/high-level-api.rst | 6 ++--- examples/CMakeLists.txt | 6 ++++- examples/Makefile | 6 ++++- examples/array.cpp | 42 ++++++++++++++++++++++++++++++++++ 7 files changed, 74 insertions(+), 7 deletions(-) create mode 100644 examples/array.cpp diff --git a/README.md b/README.md index 3ca6903ff..086c4b8a4 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,9 @@ e.g., CUDA support. For further configuration and build instructions, please consult the [documentation](https://zfp.readthedocs.io/en/latest/installation.html). +For examples of how to call the |zfp| library and use its C++ array classes, +see the [examples](https://zfp.readthedocs.io/en/latest/examples.html) +section. Documentation diff --git a/docs/source/defs.rst b/docs/source/defs.rst index 6d6a82dd3..ada0f728d 100644 --- a/docs/source/defs.rst +++ b/docs/source/defs.rst @@ -38,3 +38,4 @@ .. |fieldrelease| replace:: 0.5.6 .. |carrrelease| replace:: 0.5.6 .. |cpprelease| replace:: 0.5.6 +.. |verrelease| replace:: 0.5.6 diff --git a/docs/source/examples.rst b/docs/source/examples.rst index a6ad92a98..8fb2aae1d 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -15,8 +15,7 @@ The :program:`simple` program is a minimal example that shows how to call the compressor and decompressor on a double-precision 3D array. Without the :code:`-d` option, it will compress the array and write the compressed stream to standard output. With the :code:`-d` option, it will instead -read the compressed stream from standard input and decompress the -array:: +read the compressed stream from standard input and decompress the array:: simple > compressed.zfp simple -d < compressed.zfp @@ -24,6 +23,20 @@ array:: For a more elaborate use of the compressor, see the :ref:`zfp utility `. +.. _ex-array: + +Compressed-Array C++ Classes +---------------------------- + +The :program:`array` program shows how to declare, write to, and read from +|zfp|'s compressed-array C++ objects (in this case, 2D double-precision +arrays), which is essentially as straightforward as working with STL vectors. +This example initializes a 2D array with a linear ramp of 12 |times| 8 = 96 +values using only four bits of storage per value, which using uncompressed +storage would not be enough to distinguish more than 16 different values. +For more advanced compressed-array features, see the +:ref:`tutorial `. + .. _ex-diffusion: Diffusion Solver diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index d006baa3f..ae543275f 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -46,7 +46,7 @@ Macros (see :c:macro:`ZFP_MAKE_VERSION`). :c:macro:`ZFP_VERSION_STRING` is a string literal (see :c:macro:`ZFP_MAKE_VERSION_STRING`). See also :c:data:`zfp_library_version` and :c:data:`zfp_version_string`. - :c:macro:`ZFP_VERSION_TWEAK` is new as of |zfp| |versionrelease| and used + :c:macro:`ZFP_VERSION_TWEAK` is new as of |zfp| |verrelease| and used to mark intermediate develop versions. ---- @@ -68,7 +68,7 @@ Macros Utility macros for constructing :c:macro:`ZFP_VERSION` and :c:macro:`ZFP_VERSION_STRING`, respectively. Includes tweak version used by intermediate develop versions. Available as of - |zfp| |versionrelease|, these macros may be used by applications to test + |zfp| |verrelease|, these macros may be used by applications to test for a certain |zfp| version number, e.g., :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(0, 5, 6, 1)`. @@ -85,7 +85,7 @@ Macros Macro identifying that the current version is an intermediate develop version as opposed to an official release. Available as of |zfp| - |versionrelease|. + |verrelease|. ---- diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt index 2861e470a..73137223f 100644 --- a/examples/CMakeLists.txt +++ b/examples/CMakeLists.txt @@ -1,3 +1,7 @@ +add_executable(array array.cpp) +target_compile_definitions(array PRIVATE ${zfp_compressed_array_defs}) +target_link_libraries(array zfp) + add_executable(diffusion diffusion.cpp) target_compile_definitions(diffusion PRIVATE ${zfp_compressed_array_defs}) if(ZFP_WITH_OPENMP) @@ -6,7 +10,6 @@ else() target_link_libraries(diffusion zfp) endif() - if(BUILD_CFP) add_executable(diffusionC diffusionC.c) target_link_libraries(diffusionC cfp) @@ -38,6 +41,7 @@ add_executable(speed speed.c) target_link_libraries(speed zfp) if(HAVE_LIBM_MATH) + target_link_libraries(array m) target_link_libraries(diffusion m) if(BUILD_CFP) diff --git a/examples/Makefile b/examples/Makefile index 2dc6213f5..0c922b4cc 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -1,7 +1,8 @@ include ../Config BINDIR = ../bin -TARGETS = $(BINDIR)/diffusion\ +TARGETS = $(BINDIR)/array\ + $(BINDIR)/diffusion\ $(BINDIR)/inplace\ $(BINDIR)/iterator\ $(BINDIR)/pgm\ @@ -21,6 +22,9 @@ endif all: $(TARGETS) +$(BINDIR)/array: array.cpp ../lib/$(LIBZFP) + $(CXX) $(CXXFLAGS) $(INCS) -I../array array.cpp $(CXXLIBS) -o $@ + $(BINDIR)/diffusion: diffusion.cpp ../lib/$(LIBZFP) $(CXX) $(CXXFLAGS) $(INCS) -I../array diffusion.cpp $(CXXLIBS) -o $@ diff --git a/examples/array.cpp b/examples/array.cpp new file mode 100644 index 000000000..546de33ce --- /dev/null +++ b/examples/array.cpp @@ -0,0 +1,42 @@ +// simple example that shows how to work with zfp's compressed-array classes + +#include +#include +#include "zfparray2.h" + +int main() +{ + // array dimensions (can be arbitrary) and zfp memory footprint + const size_t nx = 12; + const size_t ny = 8; + const double bits_per_value = 4.0; + + // declare 2D arrays using STL and zfp + std::vector vec(nx * ny); + zfp::array2 arr(nx, ny, bits_per_value); + + // initialize arrays to linear ramp + for (size_t y = 0; y < ny; y++) + for (size_t x = 0; x < nx; x++) + arr(x, y) = vec[x + nx * y] = x + nx * y; + + // alternative initialization of entire array, arr: + // arr.set(&vec[0]); + + // optional: force compression of cached data + arr.flush_cache(); + + // print values + for (size_t y = 0; y < ny; y++) + for (size_t x = 0; x < nx; x++) + std::cout << vec[x + nx * y] << " " << arr(x, y) << std::endl; + + // alternative using printf(); note the necessary cast: + // printf("%g %g\n", vec[x + nx * y], (double)arr(x, y)); + + // print storage size of payload data + std::cout << "vec bytes = " << vec.capacity() * sizeof(vec[0]) << std::endl; + std::cout << "zfp bytes = " << arr.size_bytes(ZFP_DATA_PAYLOAD) << std::endl; + + return 0; +} From bb59d84574b1f3ef149ecfbd3b6de0bd13b80f35 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 30 Jun 2022 10:02:08 -0700 Subject: [PATCH 063/277] Fix minor documentation issues --- README.md | 2 +- docs/source/examples.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 086c4b8a4..50fa4595e 100644 --- a/README.md +++ b/README.md @@ -57,7 +57,7 @@ e.g., CUDA support. For further configuration and build instructions, please consult the [documentation](https://zfp.readthedocs.io/en/latest/installation.html). -For examples of how to call the |zfp| library and use its C++ array classes, +For examples of how to call the C library and use the C++ array classes, see the [examples](https://zfp.readthedocs.io/en/latest/examples.html) section. diff --git a/docs/source/examples.rst b/docs/source/examples.rst index 8fb2aae1d..786451de1 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -3,7 +3,7 @@ Code Examples ============= -The :file:`examples` directory includes five programs that make use of the +The :file:`examples` directory includes seven programs that make use of the compressor. .. _ex-simple: From 26678cdeda84be7c0f0fe027a0144d8d2bc1c210 Mon Sep 17 00:00:00 2001 From: Peter Lindstrom Date: Thu, 30 Jun 2022 10:03:56 -0700 Subject: [PATCH 064/277] Replace stdio with iostream in testviews --- tests/testviews.cpp | 105 ++++++++++++++++++++++++++------------------ 1 file changed, 63 insertions(+), 42 deletions(-) diff --git a/tests/testviews.cpp b/tests/testviews.cpp index f01b7cd16..3827c5265 100644 --- a/tests/testviews.cpp +++ b/tests/testviews.cpp @@ -1,7 +1,7 @@ -#include #include -#include #include +#include +#include #include "zfparray2.h" #include "zfparray3.h" @@ -19,47 +19,68 @@ static void verify(double f, double g) { if (std::fabs(f - g) > EPSILON) { - fprintf(stderr, "error: %g != %g\n", f, g); + std::cerr << "error: " << f << " != " << g << std::endl; exit(EXIT_FAILURE); } } +static int +usage() +{ + std::cerr << "Usage: testviews [nx ny nz [x0 y0 z0 mx my mz]]" << std::endl; + return EXIT_FAILURE; +} + int main(int argc, char* argv[]) { size_t nx = 16; size_t ny = 16; size_t nz = 16; - size_t x0 = rand(0, nx); - size_t y0 = rand(0, ny); - size_t z0 = rand(0, nz); - size_t mx = rand(1, nx - x0); - size_t my = rand(1, ny - y0); - size_t mz = rand(1, nz - z0); + size_t x0, y0, z0; + size_t mx, my, mz; double rate = 16; - // Usage: test [nx ny nz [x0 y0 z0 mx my mz]] + // parse command-line arguments switch (argc) { case 10: - if (sscanf(argv[4], "%zu", &x0) != 1 || - sscanf(argv[5], "%zu", &y0) != 1 || - sscanf(argv[6], "%zu", &z0) != 1 || - sscanf(argv[7], "%zu", &mx) != 1 || - sscanf(argv[8], "%zu", &my) != 1 || - sscanf(argv[9], "%zu", &mz) != 1) - return EXIT_FAILURE; + if ((std::istringstream(argv[4]) >> x0).fail() || + (std::istringstream(argv[5]) >> y0).fail() || + (std::istringstream(argv[6]) >> z0).fail() || + (std::istringstream(argv[7]) >> mx).fail() || !mx || + (std::istringstream(argv[8]) >> my).fail() || !my || + (std::istringstream(argv[9]) >> mz).fail() || !mz) + return usage(); // FALLTHROUGH case 4: - if (sscanf(argv[1], "%zu", &nx) != 1 || - sscanf(argv[2], "%zu", &ny) != 1 || - sscanf(argv[3], "%zu", &nz) != 1) - return EXIT_FAILURE; + if ((std::istringstream(argv[1]) >> nx).fail() || !nx || + (std::istringstream(argv[2]) >> ny).fail() || !ny || + (std::istringstream(argv[3]) >> nz).fail() || !nz) + return usage(); // FALLTHROUGH case 1: break; + default: + return usage(); + } + + if (argc < 10) { + // generate random view + x0 = rand(0, nx); + y0 = rand(0, ny); + z0 = rand(0, nz); + mx = rand(0, nx - x0); + my = rand(0, ny - y0); + mz = rand(0, nz - z0); + } + + // validate arguments + if (x0 + mx > nx || y0 + my > ny || z0 + mz > nz) { + std::cerr << "invalid view parameters" << std::endl; + return EXIT_FAILURE; } - printf("a(%zu, %zu, %zu)\n", nx, ny, nz); - printf("v(%zu, %zu, %zu) + (%zu, %zu, %zu)\n", mx, my, mz, x0, y0, z0); + std::cout << "a(" << nx << ", " << ny << ", " << nz << ")" << std::endl; + std::cout << "v(" << mx << ", " << my << ", " << mz << ") + (" << x0 << ", " << y0 << ", " << z0 << ")" << std::endl; // initialize 3D array to linear function zfp::array3 a(nx, ny, nz, rate); @@ -69,37 +90,37 @@ int main(int argc, char* argv[]) a(x, y, z) = static_cast(x + nx * (y + ny * z)); // rectangular view into a - printf("\n3D view\n"); + std::cout << std::endl << "3D view" << std::endl; zfp::array3::view v(&a, x0, y0, z0, mx, my, mz); for (size_t z = 0; z < v.size_z(); z++) for (size_t y = 0; y < v.size_y(); y++) for (size_t x = 0; x < v.size_x(); x++) { - printf("%zu %zu %zu: %g %g\n", x, y, z, (double)a(x0 + x, y0 + y, z0 + z), (double)v(x, y, z)); + std::cout << x << " " << y << " " << z << ": " << a(x0 + x, y0 + y, z0 + z) << " " << v(x, y, z) << std::endl; verify(a(x0 + x, y0 + y, z0 + z), v(x, y, z)); } // flat view of all of a - printf("\n3D flat view\n"); + std::cout << std::endl << "3D flat view" << std::endl; zfp::array3::flat_view fv(&a); for (size_t z = 0; z < fv.size_z(); z++) for (size_t y = 0; y < fv.size_y(); y++) for (size_t x = 0; x < fv.size_x(); x++) { - printf("%zu %zu %zu: %g %g\n", x, y, z, (double)a(x, y, z), (double)fv[fv.index(x, y, z)]); + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << fv[fv.index(x, y, z)] << std::endl; verify(a(x, y, z), fv[fv.index(x, y, z)]); } // nested view of all of a - printf("\n3D nested view\n"); + std::cout << std::endl << "3D nested view" << std::endl; zfp::array3::nested_view nv(&a); for (size_t z = 0; z < v.size_z(); z++) for (size_t y = 0; y < v.size_y(); y++) for (size_t x = 0; x < v.size_x(); x++) { - printf("%zu %zu %zu: %g %g\n", x, y, z, (double)a(x, y, z), (double)nv[z][y][x]); + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << nv[z][y][x] << std::endl; verify(a(x, y, z), nv[z][y][x]); } // pointers and iterators into a via view v - printf("\n3D view pointers and iterators\n"); + std::cout << std::endl << "3D view pointers and iterators" << std::endl; zfp::array3::view::const_reference vr = v(0, 0, 0); zfp::array3::view::const_pointer p = &vr; p = &v(0, 0, 0); @@ -111,7 +132,7 @@ int main(int argc, char* argv[]) } // pointers and iterators into a via flat view fv - printf("\n3D flat view pointers and iterators\n"); + std::cout << std::endl << "3D flat view pointers and iterators" << std::endl; zfp::array3::flat_view::const_reference fvr = fv[0]; zfp::array3::flat_view::const_pointer fp = &fvr; fp = &fv(0, 0, 0); @@ -123,52 +144,52 @@ int main(int argc, char* argv[]) } // 2D slice of a - printf("\n2D slice\n"); + std::cout << std::endl << "2D slice" << std::endl; size_t z = rand(0, nv.size_z()); zfp::array3::nested_view2 slice2(nv[z]); for (size_t y = 0; y < slice2.size_y(); y++) for (size_t x = 0; x < slice2.size_x(); x++) { - printf("%zu %zu %zu: %g %g\n", x, y, z, (double)a(x, y, z), (double)slice2[y][x]); + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << slice2[y][x] << std::endl; verify(a(x, y, z), slice2[y][x]); } // 2D array constructed from 2D slice (exercises deep copy via iterator) - printf("\n2D array from 2D slice\n"); + std::cout << std::endl << "2D array from 2D slice" << std::endl; zfp::array2 b(slice2); for (size_t y = 0; y < b.size_y(); y++) for (size_t x = 0; x < b.size_x(); x++) { - printf("%zu %zu: %g %g\n", x, y, (double)b(x, y), (double)slice2[y][x]); + std::cout << x << " " << y << ": " << b(x, y) << " " << slice2[y][x] << std::endl; verify(b(x, y), slice2[y][x]); } // 1D slice of a - printf("\n1D slice\n"); + std::cout << std::endl << "1D slice" << std::endl; size_t y = rand(0, slice2.size_y()); zfp::array3::nested_view1 slice1 = slice2[y]; for (size_t x = 0; x < slice1.size_x(); x++) { - printf("%zu %zu %zu: %g %g\n", x, y, z, (double)a(x, y, z), (double)slice1[x]); + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << slice1[x] << std::endl; verify(a(x, y, z), slice1[x]); } // 2D array constructed from 2D slice of 3D array (exercises deep copy via iterator) - printf("\n2D array from 2D slice of 3D array\n"); + std::cout << std::endl << "2D array from 2D slice of 3D array" << std::endl; zfp::array2 c(slice2); for (size_t y = 0; y < c.size_y(); y++) for (size_t x = 0; x < c.size_x(); x++) { - printf("%zu %zu: %g %g\n", x, y, (double)c(x, y), (double)slice2[y][x]); + std::cout << x << " " << y << ": " << c(x, y) << " " << slice2[y][x] << std::endl; verify(c(x, y), slice2[y][x]); } // 2D thread-safe view of c - printf("\n2D private view\n"); + std::cout << std::endl << "2D private view" << std::endl; zfp::array2::private_const_view d(&c); for (size_t y = 0; y < c.size_y(); y++) for (size_t x = 0; x < c.size_x(); x++) { - printf("%zu %zu: %g %g\n", x, y, (double)c(x, y), (double)d(x, y)); + std::cout << x << " " << y << ": " << c(x, y) << " " << d(x, y) << std::endl; verify(c(x, y), d(x, y)); } - printf("\nall tests passed\n"); + std::cout << std::endl << "all tests passed" << std::endl; return 0; } From 9b0d278f8efa1b070e94e49d6d47a40be6a223a9 Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Wed, 6 Jul 2022 09:36:34 -0700 Subject: [PATCH 065/277] Update where gitlab cpu tests are run --- tests/gitlab/gitlab-ci.yml | 2 + tests/gitlab/pascal-jobs.yml | 78 ------------------------------- tests/gitlab/pascal-templates.yml | 8 ---- tests/gitlab/quartz-jobs.yml | 77 ++++++++++++++++++++++++++++++ tests/gitlab/quartz-templates.yml | 12 +++++ 5 files changed, 91 insertions(+), 86 deletions(-) create mode 100644 tests/gitlab/quartz-jobs.yml create mode 100644 tests/gitlab/quartz-templates.yml diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index e7dc30654..f0980e0b9 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -120,5 +120,7 @@ stages: include: - local: tests/gitlab/pascal-templates.yml - local: tests/gitlab/pascal-jobs.yml + - local: tests/gitlab/quartz-templates.yml + - local: tests/gitlab/quartz-jobs.yml # - local: tests/gitlab/corona-templates.yml # - local: tests/gitlab/corona-jobs.yml diff --git a/tests/gitlab/pascal-jobs.yml b/tests/gitlab/pascal-jobs.yml index 8bbcaf155..3f3636558 100644 --- a/tests/gitlab/pascal-jobs.yml +++ b/tests/gitlab/pascal-jobs.yml @@ -14,81 +14,3 @@ cuda-10.1.168_test: ci_test_regex: "Cuda" extends: [.pascal_test_gpu] needs: [cuda-10.1.168_build] - -########### -# CXX CPU # -########### - -cpp_gnu-7.3.0_build: - variables: - ci_cmake: "cmake/3.9.2" - ci_cxx_cmp: "g++" - ci_c_cmp: "gcc" - ci_cmp_mod: "gcc/7.3.0" - extends: [.cpp, .pascal_build_cpu] - needs: [] - -cpp_gnu-7.3.0_test: - extends: [.pascal_test_cpu] - needs: [cpp_gnu-7.3.0_build] - - -cpp_clang-10.0.0_build: - variables: - ci_cmake: "cmake/3.9.2" - ci_cxx_cmp: "clang++" - ci_c_cmp: "clang" - ci_cmp_mod: "clang/10.0.0" - extends: [.cpp, .pascal_build_cpu] - needs: [] - -cpp_clang-10.0.0_test: - extends: [.pascal_test_cpu] - needs: [cpp_clang-10.0.0_build] - - -cpp_intel-19.0.4_build: - variables: - ci_cmake: "cmake/3.9.2" - ci_cxx_cmp: "icpc" - ci_c_cmp: "icc" - ci_cmp_mod: "intel/19.0.4" - extends: [.cpp, .pascal_build_cpu] - needs: [] - -cpp_intel-19.0.4_test: - extends: [.pascal_test_cpu] - needs: [cpp_intel-19.0.4_build] - - -cpp_pgi-21.1_build: - variables: - ci_cmake: "cmake/3.9.2" - ci_cxx_cmp: "pgc++" - ci_c_cmp: "pgcc" - ci_cmp_mod: "pgi/21.1" - extends: [.cpp, .pascal_build_cpu] - needs: [] - -cpp_pgi-21.1_test: - extends: [.pascal_test_cpu] - needs: [cpp_pgi-21.1_build] - - -######### -# C CPU # -######### - -c_gnu-7.3.0_build: - variables: - ci_cmake: "cmake/3.9.2" - ci_c_cmp: "gcc" - ci_cmp_mod: "gcc/7.3.0" - extends: [.c, .pascal_build_cpu] - needs: [] - -c_gnu-7.3.0_test: - variables: - ci_test_regex: "Cfp" - extends: [.pascal_test_cpu] - needs: [c_gnu-7.3.0_build] diff --git a/tests/gitlab/pascal-templates.yml b/tests/gitlab/pascal-templates.yml index 1537088ea..30b262696 100644 --- a/tests/gitlab/pascal-templates.yml +++ b/tests/gitlab/pascal-templates.yml @@ -10,11 +10,3 @@ variables: ci_test_regex: "." extends: [.test_gpu, .pascal_job] - -.pascal_build_cpu: - extends: [.build_cpu, .pascal_job] - -.pascal_test_cpu: - variables: - ci_test_regex: "." - extends: [.test_cpu, .pascal_job] diff --git a/tests/gitlab/quartz-jobs.yml b/tests/gitlab/quartz-jobs.yml new file mode 100644 index 000000000..071728b32 --- /dev/null +++ b/tests/gitlab/quartz-jobs.yml @@ -0,0 +1,77 @@ +########### +# CXX CPU # +########### + +cpp_gnu-7.3.0_build: + variables: + ci_cmake: "cmake/3.9.2" + ci_cxx_cmp: "g++" + ci_c_cmp: "gcc" + ci_cmp_mod: "gcc/7.3.0" + extends: [.cpp, .quartz_build_cpu] + needs: [] + +cpp_gnu-7.3.0_test: + extends: [.quartz_test_cpu] + needs: [cpp_gnu-7.3.0_build] + + +cpp_clang-10.0.0_build: + variables: + ci_cmake: "cmake/3.9.2" + ci_cxx_cmp: "clang++" + ci_c_cmp: "clang" + ci_cmp_mod: "clang/10.0.0" + extends: [.cpp, .quartz_build_cpu] + needs: [] + +cpp_clang-10.0.0_test: + extends: [.quartz_test_cpu] + needs: [cpp_clang-10.0.0_build] + + +cpp_intel-19.0.4_build: + variables: + ci_cmake: "cmake/3.9.2" + ci_cxx_cmp: "icpc" + ci_c_cmp: "icc" + ci_cmp_mod: "intel/19.0.4" + extends: [.cpp, .quartz_build_cpu] + needs: [] + +cpp_intel-19.0.4_test: + extends: [.quartz_test_cpu] + needs: [cpp_intel-19.0.4_build] + + +cpp_pgi-21.1_build: + variables: + ci_cmake: "cmake/3.9.2" + ci_cxx_cmp: "pgc++" + ci_c_cmp: "pgcc" + ci_cmp_mod: "pgi/21.1" + extends: [.cpp, .quartz_build_cpu] + needs: [] + +cpp_pgi-21.1_test: + extends: [.quartz_test_cpu] + needs: [cpp_pgi-21.1_build] + + +######### +# C CPU # +######### + +c_gnu-7.3.0_build: + variables: + ci_cmake: "cmake/3.9.2" + ci_c_cmp: "gcc" + ci_cmp_mod: "gcc/7.3.0" + extends: [.c, .quartz_build_cpu] + needs: [] + +c_gnu-7.3.0_test: + variables: + ci_test_regex: "Cfp" + extends: [.quartz_test_cpu] + needs: [c_gnu-7.3.0_build] diff --git a/tests/gitlab/quartz-templates.yml b/tests/gitlab/quartz-templates.yml new file mode 100644 index 000000000..d4d18533b --- /dev/null +++ b/tests/gitlab/quartz-templates.yml @@ -0,0 +1,12 @@ +.quartz_job: + tags: + - batch + - quartz + +.quartz_build_cpu: + extends: [.build_cpu, .quartz_job] + +.quartz_test_cpu: + variables: + ci_test_regex: "." + extends: [.test_cpu, .quartz_job] From e1e4840c09e371bda42ca5f8f805abe8737af01b Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Jul 2022 10:58:47 -0700 Subject: [PATCH 066/277] Move include files --- cfp/include/cfparrays.h | 2 -- array/zfpcpp.h => include/zfp.hpp | 0 array/zfparray.h => include/zfp/array.hpp | 0 array/zfparray1.h => include/zfp/array1.hpp | 0 array/zfparray2.h => include/zfp/array2.hpp | 0 array/zfparray3.h => include/zfp/array3.hpp | 0 array/zfparray4.h => include/zfp/array4.hpp | 0 include/{ => zfp}/bitstream.h | 0 cfp/include/cfparray.h => include/zfp/cfp.h | 0 array/gencodec.h => include/zfp/codec/generic.hpp | 0 array/zfpcodec.h => include/zfp/codec/zfp.hpp | 0 array/zfpcarray1.h => include/zfp/constarray1.h | 0 array/zfpcarray2.h => include/zfp/constarray2.hpp | 0 array/zfpcarray3.h => include/zfp/constarray3.hpp | 0 array/zfpcarray4.h => include/zfp/constarray4.hpp | 0 array/zfpfactory.h => include/zfp/factory.hpp | 0 array/zfpindex.h => include/zfp/index.hpp | 0 array/zfp/cache.h => include/zfp/internal/array/cache.hpp | 0 array/zfp/cache1.h => include/zfp/internal/array/cache1.hpp | 0 array/zfp/cache2.h => include/zfp/internal/array/cache2.hpp | 0 array/zfp/cache3.h => include/zfp/internal/array/cache3.hpp | 0 array/zfp/cache4.h => include/zfp/internal/array/cache4.hpp | 0 .../zfp/exception.h => include/zfp/internal/array/exception.hpp | 0 .../zfp/genheader.h => include/zfp/internal/array/genheader.hpp | 0 array/zfp/handle1.h => include/zfp/internal/array/handle1.hpp | 0 array/zfp/handle2.h => include/zfp/internal/array/handle2.hpp | 0 array/zfp/handle3.h => include/zfp/internal/array/handle3.hpp | 0 array/zfp/handle4.h => include/zfp/internal/array/handle4.hpp | 0 array/zfp/header.h => include/zfp/internal/array/header.hpp | 0 .../zfp/iterator1.h => include/zfp/internal/array/iterator1.hpp | 0 .../zfp/iterator2.h => include/zfp/internal/array/iterator2.hpp | 0 .../zfp/iterator3.h => include/zfp/internal/array/iterator3.hpp | 0 .../zfp/iterator4.h => include/zfp/internal/array/iterator4.hpp | 0 array/zfp/memory.h => include/zfp/internal/array/memory.hpp | 0 array/zfp/pointer1.h => include/zfp/internal/array/pointer1.hpp | 0 array/zfp/pointer2.h => include/zfp/internal/array/pointer2.hpp | 0 array/zfp/pointer3.h => include/zfp/internal/array/pointer3.hpp | 0 array/zfp/pointer4.h => include/zfp/internal/array/pointer4.hpp | 0 .../reference1.h => include/zfp/internal/array/reference1.hpp | 0 .../reference2.h => include/zfp/internal/array/reference2.hpp | 0 .../reference3.h => include/zfp/internal/array/reference3.hpp | 0 .../reference4.h => include/zfp/internal/array/reference4.hpp | 0 array/zfp/store.h => include/zfp/internal/array/store.hpp | 0 array/zfp/store1.h => include/zfp/internal/array/store1.hpp | 0 array/zfp/store2.h => include/zfp/internal/array/store2.hpp | 0 array/zfp/store3.h => include/zfp/internal/array/store3.hpp | 0 array/zfp/store4.h => include/zfp/internal/array/store4.hpp | 0 array/zfp/traits.h => include/zfp/internal/array/traits.hpp | 0 array/zfp/view1.h => include/zfp/internal/array/view1.hpp | 0 array/zfp/view2.h => include/zfp/internal/array/view2.hpp | 0 array/zfp/view3.h => include/zfp/internal/array/view3.hpp | 0 array/zfp/view4.h => include/zfp/internal/array/view4.hpp | 0 .../zfp/zfpheader.h => include/zfp/internal/array/zfpheader.hpp | 0 cfp/include/cfparray1d.h => include/zfp/internal/cfp/array1d.h | 0 cfp/include/cfparray1f.h => include/zfp/internal/cfp/array1f.h | 0 cfp/include/cfparray2d.h => include/zfp/internal/cfp/array2d.h | 0 cfp/include/cfparray2f.h => include/zfp/internal/cfp/array2f.h | 0 cfp/include/cfparray3d.h => include/zfp/internal/cfp/array3d.h | 0 cfp/include/cfparray3f.h => include/zfp/internal/cfp/array3f.h | 0 cfp/include/cfparray4d.h => include/zfp/internal/cfp/array4d.h | 0 cfp/include/cfparray4f.h => include/zfp/internal/cfp/array4f.h | 0 cfp/include/cfpheader.h => include/zfp/internal/cfp/header.h | 0 include/zfp/{ => internal/zfp}/macros.h | 0 include/zfp/{ => internal/zfp}/system.h | 0 include/zfp/{ => internal/zfp}/types.h | 0 65 files changed, 2 deletions(-) delete mode 100644 cfp/include/cfparrays.h rename array/zfpcpp.h => include/zfp.hpp (100%) rename array/zfparray.h => include/zfp/array.hpp (100%) rename array/zfparray1.h => include/zfp/array1.hpp (100%) rename array/zfparray2.h => include/zfp/array2.hpp (100%) rename array/zfparray3.h => include/zfp/array3.hpp (100%) rename array/zfparray4.h => include/zfp/array4.hpp (100%) rename include/{ => zfp}/bitstream.h (100%) rename cfp/include/cfparray.h => include/zfp/cfp.h (100%) rename array/gencodec.h => include/zfp/codec/generic.hpp (100%) rename array/zfpcodec.h => include/zfp/codec/zfp.hpp (100%) rename array/zfpcarray1.h => include/zfp/constarray1.h (100%) rename array/zfpcarray2.h => include/zfp/constarray2.hpp (100%) rename array/zfpcarray3.h => include/zfp/constarray3.hpp (100%) rename array/zfpcarray4.h => include/zfp/constarray4.hpp (100%) rename array/zfpfactory.h => include/zfp/factory.hpp (100%) rename array/zfpindex.h => include/zfp/index.hpp (100%) rename array/zfp/cache.h => include/zfp/internal/array/cache.hpp (100%) rename array/zfp/cache1.h => include/zfp/internal/array/cache1.hpp (100%) rename array/zfp/cache2.h => include/zfp/internal/array/cache2.hpp (100%) rename array/zfp/cache3.h => include/zfp/internal/array/cache3.hpp (100%) rename array/zfp/cache4.h => include/zfp/internal/array/cache4.hpp (100%) rename array/zfp/exception.h => include/zfp/internal/array/exception.hpp (100%) rename array/zfp/genheader.h => include/zfp/internal/array/genheader.hpp (100%) rename array/zfp/handle1.h => include/zfp/internal/array/handle1.hpp (100%) rename array/zfp/handle2.h => include/zfp/internal/array/handle2.hpp (100%) rename array/zfp/handle3.h => include/zfp/internal/array/handle3.hpp (100%) rename array/zfp/handle4.h => include/zfp/internal/array/handle4.hpp (100%) rename array/zfp/header.h => include/zfp/internal/array/header.hpp (100%) rename array/zfp/iterator1.h => include/zfp/internal/array/iterator1.hpp (100%) rename array/zfp/iterator2.h => include/zfp/internal/array/iterator2.hpp (100%) rename array/zfp/iterator3.h => include/zfp/internal/array/iterator3.hpp (100%) rename array/zfp/iterator4.h => include/zfp/internal/array/iterator4.hpp (100%) rename array/zfp/memory.h => include/zfp/internal/array/memory.hpp (100%) rename array/zfp/pointer1.h => include/zfp/internal/array/pointer1.hpp (100%) rename array/zfp/pointer2.h => include/zfp/internal/array/pointer2.hpp (100%) rename array/zfp/pointer3.h => include/zfp/internal/array/pointer3.hpp (100%) rename array/zfp/pointer4.h => include/zfp/internal/array/pointer4.hpp (100%) rename array/zfp/reference1.h => include/zfp/internal/array/reference1.hpp (100%) rename array/zfp/reference2.h => include/zfp/internal/array/reference2.hpp (100%) rename array/zfp/reference3.h => include/zfp/internal/array/reference3.hpp (100%) rename array/zfp/reference4.h => include/zfp/internal/array/reference4.hpp (100%) rename array/zfp/store.h => include/zfp/internal/array/store.hpp (100%) rename array/zfp/store1.h => include/zfp/internal/array/store1.hpp (100%) rename array/zfp/store2.h => include/zfp/internal/array/store2.hpp (100%) rename array/zfp/store3.h => include/zfp/internal/array/store3.hpp (100%) rename array/zfp/store4.h => include/zfp/internal/array/store4.hpp (100%) rename array/zfp/traits.h => include/zfp/internal/array/traits.hpp (100%) rename array/zfp/view1.h => include/zfp/internal/array/view1.hpp (100%) rename array/zfp/view2.h => include/zfp/internal/array/view2.hpp (100%) rename array/zfp/view3.h => include/zfp/internal/array/view3.hpp (100%) rename array/zfp/view4.h => include/zfp/internal/array/view4.hpp (100%) rename array/zfp/zfpheader.h => include/zfp/internal/array/zfpheader.hpp (100%) rename cfp/include/cfparray1d.h => include/zfp/internal/cfp/array1d.h (100%) rename cfp/include/cfparray1f.h => include/zfp/internal/cfp/array1f.h (100%) rename cfp/include/cfparray2d.h => include/zfp/internal/cfp/array2d.h (100%) rename cfp/include/cfparray2f.h => include/zfp/internal/cfp/array2f.h (100%) rename cfp/include/cfparray3d.h => include/zfp/internal/cfp/array3d.h (100%) rename cfp/include/cfparray3f.h => include/zfp/internal/cfp/array3f.h (100%) rename cfp/include/cfparray4d.h => include/zfp/internal/cfp/array4d.h (100%) rename cfp/include/cfparray4f.h => include/zfp/internal/cfp/array4f.h (100%) rename cfp/include/cfpheader.h => include/zfp/internal/cfp/header.h (100%) rename include/zfp/{ => internal/zfp}/macros.h (100%) rename include/zfp/{ => internal/zfp}/system.h (100%) rename include/zfp/{ => internal/zfp}/types.h (100%) diff --git a/cfp/include/cfparrays.h b/cfp/include/cfparrays.h deleted file mode 100644 index 39b267243..000000000 --- a/cfp/include/cfparrays.h +++ /dev/null @@ -1,2 +0,0 @@ -#warning "deprecated header; use cfparray.h" -#include "cfparray.h" diff --git a/array/zfpcpp.h b/include/zfp.hpp similarity index 100% rename from array/zfpcpp.h rename to include/zfp.hpp diff --git a/array/zfparray.h b/include/zfp/array.hpp similarity index 100% rename from array/zfparray.h rename to include/zfp/array.hpp diff --git a/array/zfparray1.h b/include/zfp/array1.hpp similarity index 100% rename from array/zfparray1.h rename to include/zfp/array1.hpp diff --git a/array/zfparray2.h b/include/zfp/array2.hpp similarity index 100% rename from array/zfparray2.h rename to include/zfp/array2.hpp diff --git a/array/zfparray3.h b/include/zfp/array3.hpp similarity index 100% rename from array/zfparray3.h rename to include/zfp/array3.hpp diff --git a/array/zfparray4.h b/include/zfp/array4.hpp similarity index 100% rename from array/zfparray4.h rename to include/zfp/array4.hpp diff --git a/include/bitstream.h b/include/zfp/bitstream.h similarity index 100% rename from include/bitstream.h rename to include/zfp/bitstream.h diff --git a/cfp/include/cfparray.h b/include/zfp/cfp.h similarity index 100% rename from cfp/include/cfparray.h rename to include/zfp/cfp.h diff --git a/array/gencodec.h b/include/zfp/codec/generic.hpp similarity index 100% rename from array/gencodec.h rename to include/zfp/codec/generic.hpp diff --git a/array/zfpcodec.h b/include/zfp/codec/zfp.hpp similarity index 100% rename from array/zfpcodec.h rename to include/zfp/codec/zfp.hpp diff --git a/array/zfpcarray1.h b/include/zfp/constarray1.h similarity index 100% rename from array/zfpcarray1.h rename to include/zfp/constarray1.h diff --git a/array/zfpcarray2.h b/include/zfp/constarray2.hpp similarity index 100% rename from array/zfpcarray2.h rename to include/zfp/constarray2.hpp diff --git a/array/zfpcarray3.h b/include/zfp/constarray3.hpp similarity index 100% rename from array/zfpcarray3.h rename to include/zfp/constarray3.hpp diff --git a/array/zfpcarray4.h b/include/zfp/constarray4.hpp similarity index 100% rename from array/zfpcarray4.h rename to include/zfp/constarray4.hpp diff --git a/array/zfpfactory.h b/include/zfp/factory.hpp similarity index 100% rename from array/zfpfactory.h rename to include/zfp/factory.hpp diff --git a/array/zfpindex.h b/include/zfp/index.hpp similarity index 100% rename from array/zfpindex.h rename to include/zfp/index.hpp diff --git a/array/zfp/cache.h b/include/zfp/internal/array/cache.hpp similarity index 100% rename from array/zfp/cache.h rename to include/zfp/internal/array/cache.hpp diff --git a/array/zfp/cache1.h b/include/zfp/internal/array/cache1.hpp similarity index 100% rename from array/zfp/cache1.h rename to include/zfp/internal/array/cache1.hpp diff --git a/array/zfp/cache2.h b/include/zfp/internal/array/cache2.hpp similarity index 100% rename from array/zfp/cache2.h rename to include/zfp/internal/array/cache2.hpp diff --git a/array/zfp/cache3.h b/include/zfp/internal/array/cache3.hpp similarity index 100% rename from array/zfp/cache3.h rename to include/zfp/internal/array/cache3.hpp diff --git a/array/zfp/cache4.h b/include/zfp/internal/array/cache4.hpp similarity index 100% rename from array/zfp/cache4.h rename to include/zfp/internal/array/cache4.hpp diff --git a/array/zfp/exception.h b/include/zfp/internal/array/exception.hpp similarity index 100% rename from array/zfp/exception.h rename to include/zfp/internal/array/exception.hpp diff --git a/array/zfp/genheader.h b/include/zfp/internal/array/genheader.hpp similarity index 100% rename from array/zfp/genheader.h rename to include/zfp/internal/array/genheader.hpp diff --git a/array/zfp/handle1.h b/include/zfp/internal/array/handle1.hpp similarity index 100% rename from array/zfp/handle1.h rename to include/zfp/internal/array/handle1.hpp diff --git a/array/zfp/handle2.h b/include/zfp/internal/array/handle2.hpp similarity index 100% rename from array/zfp/handle2.h rename to include/zfp/internal/array/handle2.hpp diff --git a/array/zfp/handle3.h b/include/zfp/internal/array/handle3.hpp similarity index 100% rename from array/zfp/handle3.h rename to include/zfp/internal/array/handle3.hpp diff --git a/array/zfp/handle4.h b/include/zfp/internal/array/handle4.hpp similarity index 100% rename from array/zfp/handle4.h rename to include/zfp/internal/array/handle4.hpp diff --git a/array/zfp/header.h b/include/zfp/internal/array/header.hpp similarity index 100% rename from array/zfp/header.h rename to include/zfp/internal/array/header.hpp diff --git a/array/zfp/iterator1.h b/include/zfp/internal/array/iterator1.hpp similarity index 100% rename from array/zfp/iterator1.h rename to include/zfp/internal/array/iterator1.hpp diff --git a/array/zfp/iterator2.h b/include/zfp/internal/array/iterator2.hpp similarity index 100% rename from array/zfp/iterator2.h rename to include/zfp/internal/array/iterator2.hpp diff --git a/array/zfp/iterator3.h b/include/zfp/internal/array/iterator3.hpp similarity index 100% rename from array/zfp/iterator3.h rename to include/zfp/internal/array/iterator3.hpp diff --git a/array/zfp/iterator4.h b/include/zfp/internal/array/iterator4.hpp similarity index 100% rename from array/zfp/iterator4.h rename to include/zfp/internal/array/iterator4.hpp diff --git a/array/zfp/memory.h b/include/zfp/internal/array/memory.hpp similarity index 100% rename from array/zfp/memory.h rename to include/zfp/internal/array/memory.hpp diff --git a/array/zfp/pointer1.h b/include/zfp/internal/array/pointer1.hpp similarity index 100% rename from array/zfp/pointer1.h rename to include/zfp/internal/array/pointer1.hpp diff --git a/array/zfp/pointer2.h b/include/zfp/internal/array/pointer2.hpp similarity index 100% rename from array/zfp/pointer2.h rename to include/zfp/internal/array/pointer2.hpp diff --git a/array/zfp/pointer3.h b/include/zfp/internal/array/pointer3.hpp similarity index 100% rename from array/zfp/pointer3.h rename to include/zfp/internal/array/pointer3.hpp diff --git a/array/zfp/pointer4.h b/include/zfp/internal/array/pointer4.hpp similarity index 100% rename from array/zfp/pointer4.h rename to include/zfp/internal/array/pointer4.hpp diff --git a/array/zfp/reference1.h b/include/zfp/internal/array/reference1.hpp similarity index 100% rename from array/zfp/reference1.h rename to include/zfp/internal/array/reference1.hpp diff --git a/array/zfp/reference2.h b/include/zfp/internal/array/reference2.hpp similarity index 100% rename from array/zfp/reference2.h rename to include/zfp/internal/array/reference2.hpp diff --git a/array/zfp/reference3.h b/include/zfp/internal/array/reference3.hpp similarity index 100% rename from array/zfp/reference3.h rename to include/zfp/internal/array/reference3.hpp diff --git a/array/zfp/reference4.h b/include/zfp/internal/array/reference4.hpp similarity index 100% rename from array/zfp/reference4.h rename to include/zfp/internal/array/reference4.hpp diff --git a/array/zfp/store.h b/include/zfp/internal/array/store.hpp similarity index 100% rename from array/zfp/store.h rename to include/zfp/internal/array/store.hpp diff --git a/array/zfp/store1.h b/include/zfp/internal/array/store1.hpp similarity index 100% rename from array/zfp/store1.h rename to include/zfp/internal/array/store1.hpp diff --git a/array/zfp/store2.h b/include/zfp/internal/array/store2.hpp similarity index 100% rename from array/zfp/store2.h rename to include/zfp/internal/array/store2.hpp diff --git a/array/zfp/store3.h b/include/zfp/internal/array/store3.hpp similarity index 100% rename from array/zfp/store3.h rename to include/zfp/internal/array/store3.hpp diff --git a/array/zfp/store4.h b/include/zfp/internal/array/store4.hpp similarity index 100% rename from array/zfp/store4.h rename to include/zfp/internal/array/store4.hpp diff --git a/array/zfp/traits.h b/include/zfp/internal/array/traits.hpp similarity index 100% rename from array/zfp/traits.h rename to include/zfp/internal/array/traits.hpp diff --git a/array/zfp/view1.h b/include/zfp/internal/array/view1.hpp similarity index 100% rename from array/zfp/view1.h rename to include/zfp/internal/array/view1.hpp diff --git a/array/zfp/view2.h b/include/zfp/internal/array/view2.hpp similarity index 100% rename from array/zfp/view2.h rename to include/zfp/internal/array/view2.hpp diff --git a/array/zfp/view3.h b/include/zfp/internal/array/view3.hpp similarity index 100% rename from array/zfp/view3.h rename to include/zfp/internal/array/view3.hpp diff --git a/array/zfp/view4.h b/include/zfp/internal/array/view4.hpp similarity index 100% rename from array/zfp/view4.h rename to include/zfp/internal/array/view4.hpp diff --git a/array/zfp/zfpheader.h b/include/zfp/internal/array/zfpheader.hpp similarity index 100% rename from array/zfp/zfpheader.h rename to include/zfp/internal/array/zfpheader.hpp diff --git a/cfp/include/cfparray1d.h b/include/zfp/internal/cfp/array1d.h similarity index 100% rename from cfp/include/cfparray1d.h rename to include/zfp/internal/cfp/array1d.h diff --git a/cfp/include/cfparray1f.h b/include/zfp/internal/cfp/array1f.h similarity index 100% rename from cfp/include/cfparray1f.h rename to include/zfp/internal/cfp/array1f.h diff --git a/cfp/include/cfparray2d.h b/include/zfp/internal/cfp/array2d.h similarity index 100% rename from cfp/include/cfparray2d.h rename to include/zfp/internal/cfp/array2d.h diff --git a/cfp/include/cfparray2f.h b/include/zfp/internal/cfp/array2f.h similarity index 100% rename from cfp/include/cfparray2f.h rename to include/zfp/internal/cfp/array2f.h diff --git a/cfp/include/cfparray3d.h b/include/zfp/internal/cfp/array3d.h similarity index 100% rename from cfp/include/cfparray3d.h rename to include/zfp/internal/cfp/array3d.h diff --git a/cfp/include/cfparray3f.h b/include/zfp/internal/cfp/array3f.h similarity index 100% rename from cfp/include/cfparray3f.h rename to include/zfp/internal/cfp/array3f.h diff --git a/cfp/include/cfparray4d.h b/include/zfp/internal/cfp/array4d.h similarity index 100% rename from cfp/include/cfparray4d.h rename to include/zfp/internal/cfp/array4d.h diff --git a/cfp/include/cfparray4f.h b/include/zfp/internal/cfp/array4f.h similarity index 100% rename from cfp/include/cfparray4f.h rename to include/zfp/internal/cfp/array4f.h diff --git a/cfp/include/cfpheader.h b/include/zfp/internal/cfp/header.h similarity index 100% rename from cfp/include/cfpheader.h rename to include/zfp/internal/cfp/header.h diff --git a/include/zfp/macros.h b/include/zfp/internal/zfp/macros.h similarity index 100% rename from include/zfp/macros.h rename to include/zfp/internal/zfp/macros.h diff --git a/include/zfp/system.h b/include/zfp/internal/zfp/system.h similarity index 100% rename from include/zfp/system.h rename to include/zfp/internal/zfp/system.h diff --git a/include/zfp/types.h b/include/zfp/internal/zfp/types.h similarity index 100% rename from include/zfp/types.h rename to include/zfp/internal/zfp/types.h From 013210b068b272c96c4ab0176747ba6a45c14235 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Jul 2022 11:44:15 -0700 Subject: [PATCH 067/277] Update includes for all non-internal headers --- include/zfp.h | 6 +++--- include/zfp/array.hpp | 4 ++-- include/zfp/array1.hpp | 20 +++++++++---------- include/zfp/array2.hpp | 20 +++++++++---------- include/zfp/array3.hpp | 20 +++++++++---------- include/zfp/array4.hpp | 20 +++++++++---------- include/zfp/bitstream.h | 4 ++-- include/zfp/cfp.h | 18 ++++++++--------- include/zfp/codec/generic.hpp | 6 +++--- include/zfp/codec/zfp.hpp | 6 +++--- .../zfp/{constarray1.h => constarray1.hpp} | 20 +++++++++---------- include/zfp/constarray2.hpp | 20 +++++++++---------- include/zfp/constarray3.hpp | 20 +++++++++---------- include/zfp/constarray4.hpp | 20 +++++++++---------- include/zfp/factory.hpp | 12 +++++------ 15 files changed, 108 insertions(+), 108 deletions(-) rename include/zfp/{constarray1.h => constarray1.hpp} (94%) diff --git a/include/zfp.h b/include/zfp.h index 089ab94b6..2ba262bef 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -7,10 +7,10 @@ #ifndef ZFP_H #define ZFP_H -#include "zfp/types.h" -#include "zfp/system.h" +#include "zfp/bitstream.h" #include "zfp/version.h" -#include "bitstream.h" +#include "zfp/internal/zfp/system.h" +#include "zfp/internal/zfp/types.h" /* macros ------------------------------------------------------------------ */ diff --git a/include/zfp/array.hpp b/include/zfp/array.hpp index 157207b41..113c61f9b 100644 --- a/include/zfp/array.hpp +++ b/include/zfp/array.hpp @@ -5,14 +5,14 @@ #include #include #include "zfp.h" -#include "zfp/exception.h" +#include "zfp/internal/array/exception.hpp" namespace zfp { // abstract base class for compressed array of scalars class array { public: - #include "zfp/header.h" + #include "zfp/internal/array/header.hpp" // factory function (see zfpfactory.h) static zfp::array* construct(const zfp::array::header& header, const void* buffer = 0, size_t buffer_size_bytes = 0); diff --git a/include/zfp/array1.hpp b/include/zfp/array1.hpp index a388fc4cf..437f96997 100644 --- a/include/zfp/array1.hpp +++ b/include/zfp/array1.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache1.h" -#include "zfp/store1.h" -#include "zfp/handle1.h" -#include "zfp/reference1.h" -#include "zfp/pointer1.h" -#include "zfp/iterator1.h" -#include "zfp/view1.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache1.hpp" +#include "zfp/internal/array/handle1.hpp" +#include "zfp/internal/array/iterator1.hpp" +#include "zfp/internal/array/pointer1.hpp" +#include "zfp/internal/array/reference1.hpp" +#include "zfp/internal/array/store1.hpp" +#include "zfp/internal/array/view1.hpp" namespace zfp { diff --git a/include/zfp/array2.hpp b/include/zfp/array2.hpp index 7fc285cde..439a05169 100644 --- a/include/zfp/array2.hpp +++ b/include/zfp/array2.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache2.h" -#include "zfp/store2.h" -#include "zfp/handle2.h" -#include "zfp/reference2.h" -#include "zfp/pointer2.h" -#include "zfp/iterator2.h" -#include "zfp/view2.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache2.hpp" +#include "zfp/internal/array/handle2.hpp" +#include "zfp/internal/array/iterator2.hpp" +#include "zfp/internal/array/pointer2.hpp" +#include "zfp/internal/array/reference2.hpp" +#include "zfp/internal/array/store2.hpp" +#include "zfp/internal/array/view2.hpp" namespace zfp { diff --git a/include/zfp/array3.hpp b/include/zfp/array3.hpp index 8d5a6a853..ee25d95b5 100644 --- a/include/zfp/array3.hpp +++ b/include/zfp/array3.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache3.h" -#include "zfp/store3.h" -#include "zfp/handle3.h" -#include "zfp/reference3.h" -#include "zfp/pointer3.h" -#include "zfp/iterator3.h" -#include "zfp/view3.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache3.hpp" +#include "zfp/internal/array/handle3.hpp" +#include "zfp/internal/array/iterator3.hpp" +#include "zfp/internal/array/pointer3.hpp" +#include "zfp/internal/array/reference3.hpp" +#include "zfp/internal/array/store3.hpp" +#include "zfp/internal/array/view3.hpp" namespace zfp { diff --git a/include/zfp/array4.hpp b/include/zfp/array4.hpp index dbebb9015..4d95d1413 100644 --- a/include/zfp/array4.hpp +++ b/include/zfp/array4.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache4.h" -#include "zfp/store4.h" -#include "zfp/handle4.h" -#include "zfp/reference4.h" -#include "zfp/pointer4.h" -#include "zfp/iterator4.h" -#include "zfp/view4.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache4.hpp" +#include "zfp/internal/array/handle4.hpp" +#include "zfp/internal/array/iterator4.hpp" +#include "zfp/internal/array/pointer4.hpp" +#include "zfp/internal/array/reference4.hpp" +#include "zfp/internal/array/store4.hpp" +#include "zfp/internal/array/view4.hpp" namespace zfp { diff --git a/include/zfp/bitstream.h b/include/zfp/bitstream.h index f9b292186..85922aadb 100644 --- a/include/zfp/bitstream.h +++ b/include/zfp/bitstream.h @@ -2,8 +2,8 @@ #define ZFP_BITSTREAM_H #include -#include "zfp/types.h" -#include "zfp/system.h" +#include "zfp/internal/zfp/types.h" +#include "zfp/internal/zfp/system.h" /* forward declaration of opaque type */ typedef struct bitstream bitstream; diff --git a/include/zfp/cfp.h b/include/zfp/cfp.h index 5ced8b7c5..9d91a1731 100644 --- a/include/zfp/cfp.h +++ b/include/zfp/cfp.h @@ -2,15 +2,15 @@ #define CFP_ARRAY #include -#include "cfpheader.h" -#include "cfparray1f.h" -#include "cfparray1d.h" -#include "cfparray2f.h" -#include "cfparray2d.h" -#include "cfparray3f.h" -#include "cfparray3d.h" -#include "cfparray4f.h" -#include "cfparray4d.h" +#include "zfp/internal/cfp/header.h" +#include "zfp/internal/cfp/array1f.h" +#include "zfp/internal/cfp/array1d.h" +#include "zfp/internal/cfp/array2f.h" +#include "zfp/internal/cfp/array2d.h" +#include "zfp/internal/cfp/array3f.h" +#include "zfp/internal/cfp/array3d.h" +#include "zfp/internal/cfp/array4f.h" +#include "zfp/internal/cfp/array4d.h" typedef struct { cfp_array1f_api array1f; diff --git a/include/zfp/codec/generic.hpp b/include/zfp/codec/generic.hpp index 7557c2388..4363b899a 100644 --- a/include/zfp/codec/generic.hpp +++ b/include/zfp/codec/generic.hpp @@ -16,9 +16,9 @@ #include #include #include "zfp.h" -#include "zfparray.h" -#include "zfp/memory.h" -#include "zfp/traits.h" +#include "zfp/array.hpp" +#include "zfp/internal/array/memory.h" +#include "zfp/internal/array/traits.h" namespace zfp { namespace codec { diff --git a/include/zfp/codec/zfp.hpp b/include/zfp/codec/zfp.hpp index 989de59a6..d04067d9d 100644 --- a/include/zfp/codec/zfp.hpp +++ b/include/zfp/codec/zfp.hpp @@ -5,9 +5,9 @@ #include #include #include "zfp.h" -#include "zfpcpp.h" -#include "zfp/memory.h" -#include "zfp/traits.h" +#include "zfp.hpp" +#include "zfp/internal/array/memory.h" +#include "zfp/internal/array/traits.h" namespace zfp { namespace codec { diff --git a/include/zfp/constarray1.h b/include/zfp/constarray1.hpp similarity index 94% rename from include/zfp/constarray1.h rename to include/zfp/constarray1.hpp index e77015f63..6081b17f4 100644 --- a/include/zfp/constarray1.h +++ b/include/zfp/constarray1.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache1.h" -#include "zfp/store1.h" -#include "zfp/handle1.h" -#include "zfp/reference1.h" -#include "zfp/pointer1.h" -#include "zfp/iterator1.h" -#include "zfp/view1.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache1.hpp" +#include "zfp/internal/array/handle1.hpp" +#include "zfp/internal/array/iterator1.hpp" +#include "zfp/internal/array/pointer1.hpp" +#include "zfp/internal/array/reference1.hpp" +#include "zfp/internal/array/store1.hpp" +#include "zfp/internal/array/view1.hpp" namespace zfp { diff --git a/include/zfp/constarray2.hpp b/include/zfp/constarray2.hpp index 7da5d66d6..81a7eacb4 100644 --- a/include/zfp/constarray2.hpp +++ b/include/zfp/constarray2.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache2.h" -#include "zfp/store2.h" -#include "zfp/handle2.h" -#include "zfp/reference2.h" -#include "zfp/pointer2.h" -#include "zfp/iterator2.h" -#include "zfp/view2.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache2.hpp" +#include "zfp/internal/array/handle2.hpp" +#include "zfp/internal/array/iterator2.hpp" +#include "zfp/internal/array/pointer2.hpp" +#include "zfp/internal/array/reference2.hpp" +#include "zfp/internal/array/store2.hpp" +#include "zfp/internal/array/view2.hpp" namespace zfp { diff --git a/include/zfp/constarray3.hpp b/include/zfp/constarray3.hpp index bdfe21635..96ebd7fc9 100644 --- a/include/zfp/constarray3.hpp +++ b/include/zfp/constarray3.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache3.h" -#include "zfp/store3.h" -#include "zfp/handle3.h" -#include "zfp/reference3.h" -#include "zfp/pointer3.h" -#include "zfp/iterator3.h" -#include "zfp/view3.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/cache3.hpp" +#include "zfp/handle3.hpp" +#include "zfp/iterator3.hpp" +#include "zfp/pointer3.hpp" +#include "zfp/reference3.hpp" +#include "zfp/store3.hpp" +#include "zfp/view3.hpp" namespace zfp { diff --git a/include/zfp/constarray4.hpp b/include/zfp/constarray4.hpp index b556a3876..2b0159ed7 100644 --- a/include/zfp/constarray4.hpp +++ b/include/zfp/constarray4.hpp @@ -4,16 +4,16 @@ #include #include #include -#include "zfparray.h" -#include "zfpcodec.h" -#include "zfpindex.h" -#include "zfp/cache4.h" -#include "zfp/store4.h" -#include "zfp/handle4.h" -#include "zfp/reference4.h" -#include "zfp/pointer4.h" -#include "zfp/iterator4.h" -#include "zfp/view4.h" +#include "zfp/array.hpp" +#include "zfp/index.hpp" +#include "zfp/codec/zfp.hpp" +#include "zfp/internal/array/cache4.hpp" +#include "zfp/internal/array/handle4.hpp" +#include "zfp/internal/array/iterator4.hpp" +#include "zfp/internal/array/pointer4.hpp" +#include "zfp/internal/array/reference4.hpp" +#include "zfp/internal/array/store4.hpp" +#include "zfp/internal/array/view4.hpp" namespace zfp { diff --git a/include/zfp/factory.hpp b/include/zfp/factory.hpp index 972babc88..5cbdaf82d 100644 --- a/include/zfp/factory.hpp +++ b/include/zfp/factory.hpp @@ -1,9 +1,9 @@ #ifndef ZFP_FACTORY_H #define ZFP_FACTORY_H -// ensure zfparray.h has already been included +// ensure zfp/array.h has already been included #ifndef ZFP_ARRAY_H - #error "zfparray.h must be included before zfpfactory.h" + #error "zfp/array.hpp must be included before zfp/factory.hpp" #endif zfp::array* zfp::array::construct(const zfp::array::header& header, const void* buffer, size_t buffer_size_bytes) @@ -36,7 +36,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; } #else - error = "zfparray4 not supported; include zfparray4.h before zfpfactory.h"; + error = "array4 not supported; include zfp/array4.hpp before zfp/factory.hpp"; #endif break; @@ -55,7 +55,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; } #else - error = "zfparray3 not supported; include zfparray3.h before zfpfactory.h"; + error = "array3 not supported; include zfp/array3.hpp before zfp/factory.hpp"; #endif break; @@ -74,7 +74,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; } #else - error = "zfparray2 not supported; include zfparray2.h before zfpfactory.h"; + error = "array2 not supported; include zfp/array2.hpp before zfp/factory.hpp"; #endif break; @@ -93,7 +93,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; } #else - error = "zfparray1 not supported; include zfparray1.h before zfpfactory.h"; + error = "array1 not supported; include zfp/array1.hpp before zfp/factory.hpp"; #endif break; From afe6a01f926910199bb1abeacd3456db51b60b24 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Jul 2022 12:16:17 -0700 Subject: [PATCH 068/277] Update includes for internal headers --- include/zfp/internal/array/cache.hpp | 6 +++--- include/zfp/internal/array/cache1.hpp | 6 +++--- include/zfp/internal/array/cache2.hpp | 6 +++--- include/zfp/internal/array/cache3.hpp | 6 +++--- include/zfp/internal/array/cache4.hpp | 6 +++--- include/zfp/internal/array/exception.hpp | 4 ++-- include/zfp/internal/array/handle1.hpp | 4 ++-- include/zfp/internal/array/handle2.hpp | 4 ++-- include/zfp/internal/array/handle3.hpp | 4 ++-- include/zfp/internal/array/handle4.hpp | 4 ++-- include/zfp/internal/array/iterator1.hpp | 4 ++-- include/zfp/internal/array/iterator2.hpp | 4 ++-- include/zfp/internal/array/iterator3.hpp | 4 ++-- include/zfp/internal/array/iterator4.hpp | 4 ++-- include/zfp/internal/array/memory.hpp | 4 ++-- include/zfp/internal/array/pointer1.hpp | 4 ++-- include/zfp/internal/array/pointer2.hpp | 4 ++-- include/zfp/internal/array/pointer3.hpp | 4 ++-- include/zfp/internal/array/pointer4.hpp | 4 ++-- include/zfp/internal/array/reference1.hpp | 4 ++-- include/zfp/internal/array/reference2.hpp | 4 ++-- include/zfp/internal/array/reference3.hpp | 4 ++-- include/zfp/internal/array/reference4.hpp | 4 ++-- include/zfp/internal/array/store.hpp | 6 +++--- include/zfp/internal/array/store1.hpp | 6 +++--- include/zfp/internal/array/store2.hpp | 6 +++--- include/zfp/internal/array/store3.hpp | 6 +++--- include/zfp/internal/array/store4.hpp | 6 +++--- include/zfp/internal/array/traits.hpp | 4 ++-- include/zfp/internal/array/view1.hpp | 4 ++-- include/zfp/internal/array/view2.hpp | 4 ++-- include/zfp/internal/array/view3.hpp | 4 ++-- include/zfp/internal/array/view4.hpp | 4 ++-- 33 files changed, 76 insertions(+), 76 deletions(-) diff --git a/include/zfp/internal/array/cache.hpp b/include/zfp/internal/array/cache.hpp index 55b368b13..533c37dbb 100644 --- a/include/zfp/internal/array/cache.hpp +++ b/include/zfp/internal/array/cache.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_CACHE_H -#define ZFP_CACHE_H +#ifndef ZFP_CACHE_HPP +#define ZFP_CACHE_HPP -#include "memory.h" +#include "zfp/internal/array/memory.hpp" #ifdef ZFP_WITH_CACHE_PROFILE // maintain stats on hit and miss rates diff --git a/include/zfp/internal/array/cache1.hpp b/include/zfp/internal/array/cache1.hpp index 98e1791fe..24f192e5a 100644 --- a/include/zfp/internal/array/cache1.hpp +++ b/include/zfp/internal/array/cache1.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_CACHE1_H -#define ZFP_CACHE1_H +#ifndef ZFP_CACHE1_HPP +#define ZFP_CACHE1_HPP -#include "cache.h" +#include "zfp/internal/array/cache.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/cache2.hpp b/include/zfp/internal/array/cache2.hpp index 332c51707..e7aa07d90 100644 --- a/include/zfp/internal/array/cache2.hpp +++ b/include/zfp/internal/array/cache2.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_CACHE2_H -#define ZFP_CACHE2_H +#ifndef ZFP_CACHE2_HPP +#define ZFP_CACHE2_HPP -#include "cache.h" +#include "zfp/internal/array/cache.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/cache3.hpp b/include/zfp/internal/array/cache3.hpp index 95258fd8d..1c4c95544 100644 --- a/include/zfp/internal/array/cache3.hpp +++ b/include/zfp/internal/array/cache3.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_CACHE3_H -#define ZFP_CACHE3_H +#ifndef ZFP_CACHE3_HPP +#define ZFP_CACHE3_HPP -#include "cache.h" +#include "zfp/internal/array/cache.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/cache4.hpp b/include/zfp/internal/array/cache4.hpp index f16746858..69182b7ec 100644 --- a/include/zfp/internal/array/cache4.hpp +++ b/include/zfp/internal/array/cache4.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_CACHE4_H -#define ZFP_CACHE4_H +#ifndef ZFP_CACHE4_HPP +#define ZFP_CACHE4_HPP -#include "cache.h" +#include "zfp/internal/array/cache.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/exception.hpp b/include/zfp/internal/array/exception.hpp index c5c3114ea..747bf6bd6 100644 --- a/include/zfp/internal/array/exception.hpp +++ b/include/zfp/internal/array/exception.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_EXCEPTION_H -#define ZFP_EXCEPTION_H +#ifndef ZFP_EXCEPTION_HPP +#define ZFP_EXCEPTION_HPP #include #include diff --git a/include/zfp/internal/array/handle1.hpp b/include/zfp/internal/array/handle1.hpp index 4174fad69..dc692940b 100644 --- a/include/zfp/internal/array/handle1.hpp +++ b/include/zfp/internal/array/handle1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_HANDLE1_H -#define ZFP_HANDLE1_H +#ifndef ZFP_HANDLE1_HPP +#define ZFP_HANDLE1_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/handle2.hpp b/include/zfp/internal/array/handle2.hpp index 81feead11..6fa60515d 100644 --- a/include/zfp/internal/array/handle2.hpp +++ b/include/zfp/internal/array/handle2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_HANDLE2_H -#define ZFP_HANDLE2_H +#ifndef ZFP_HANDLE2_HPP +#define ZFP_HANDLE2_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/handle3.hpp b/include/zfp/internal/array/handle3.hpp index 85e09d5cf..b1fbf06dd 100644 --- a/include/zfp/internal/array/handle3.hpp +++ b/include/zfp/internal/array/handle3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_HANDLE3_H -#define ZFP_HANDLE3_H +#ifndef ZFP_HANDLE3_HPP +#define ZFP_HANDLE3_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/handle4.hpp b/include/zfp/internal/array/handle4.hpp index 3616344f6..349172a76 100644 --- a/include/zfp/internal/array/handle4.hpp +++ b/include/zfp/internal/array/handle4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_HANDLE4_H -#define ZFP_HANDLE4_H +#ifndef ZFP_HANDLE4_HPP +#define ZFP_HANDLE4_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/iterator1.hpp b/include/zfp/internal/array/iterator1.hpp index f3eb206d2..3bd0c4167 100644 --- a/include/zfp/internal/array/iterator1.hpp +++ b/include/zfp/internal/array/iterator1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ITERATOR1_H -#define ZFP_ITERATOR1_H +#ifndef ZFP_ITERATOR1_HPP +#define ZFP_ITERATOR1_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/iterator2.hpp b/include/zfp/internal/array/iterator2.hpp index bf0f674f7..133957195 100644 --- a/include/zfp/internal/array/iterator2.hpp +++ b/include/zfp/internal/array/iterator2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ITERATOR2_H -#define ZFP_ITERATOR2_H +#ifndef ZFP_ITERATOR2_HPP +#define ZFP_ITERATOR2_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/iterator3.hpp b/include/zfp/internal/array/iterator3.hpp index 386ef26cf..f733d3066 100644 --- a/include/zfp/internal/array/iterator3.hpp +++ b/include/zfp/internal/array/iterator3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ITERATOR3_H -#define ZFP_ITERATOR3_H +#ifndef ZFP_ITERATOR3_HPP +#define ZFP_ITERATOR3_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/iterator4.hpp b/include/zfp/internal/array/iterator4.hpp index d4530e42e..85253cd32 100644 --- a/include/zfp/internal/array/iterator4.hpp +++ b/include/zfp/internal/array/iterator4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ITERATOR4_H -#define ZFP_ITERATOR4_H +#ifndef ZFP_ITERATOR4_HPP +#define ZFP_ITERATOR4_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/memory.hpp b/include/zfp/internal/array/memory.hpp index d9067adf3..b6e7b9f6f 100644 --- a/include/zfp/internal/array/memory.hpp +++ b/include/zfp/internal/array/memory.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_MEMORY_H -#define ZFP_MEMORY_H +#ifndef ZFP_MEMORY_HPP +#define ZFP_MEMORY_HPP // Memory management for POD types only. Templated functions are provided only // to avoid the need for casts to/from void* in pass-by-reference calls. diff --git a/include/zfp/internal/array/pointer1.hpp b/include/zfp/internal/array/pointer1.hpp index 45fdd90c4..8c112c3cc 100644 --- a/include/zfp/internal/array/pointer1.hpp +++ b/include/zfp/internal/array/pointer1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_POINTER1_H -#define ZFP_POINTER1_H +#ifndef ZFP_POINTER1_HPP +#define ZFP_POINTER1_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/pointer2.hpp b/include/zfp/internal/array/pointer2.hpp index d025b1464..693b94d6e 100644 --- a/include/zfp/internal/array/pointer2.hpp +++ b/include/zfp/internal/array/pointer2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_POINTER2_H -#define ZFP_POINTER2_H +#ifndef ZFP_POINTER2_HPP +#define ZFP_POINTER2_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/pointer3.hpp b/include/zfp/internal/array/pointer3.hpp index 00e3ae8e5..8501b5453 100644 --- a/include/zfp/internal/array/pointer3.hpp +++ b/include/zfp/internal/array/pointer3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_POINTER3_H -#define ZFP_POINTER3_H +#ifndef ZFP_POINTER3_HPP +#define ZFP_POINTER3_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/pointer4.hpp b/include/zfp/internal/array/pointer4.hpp index 5bbc2ba35..b7f305956 100644 --- a/include/zfp/internal/array/pointer4.hpp +++ b/include/zfp/internal/array/pointer4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_POINTER4_H -#define ZFP_POINTER4_H +#ifndef ZFP_POINTER4_HPP +#define ZFP_POINTER4_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/reference1.hpp b/include/zfp/internal/array/reference1.hpp index e4e32e0bb..e41cc8b5a 100644 --- a/include/zfp/internal/array/reference1.hpp +++ b/include/zfp/internal/array/reference1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_REFERENCE1_H -#define ZFP_REFERENCE1_H +#ifndef ZFP_REFERENCE1_HPP +#define ZFP_REFERENCE1_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/reference2.hpp b/include/zfp/internal/array/reference2.hpp index 489581b06..b16484fbe 100644 --- a/include/zfp/internal/array/reference2.hpp +++ b/include/zfp/internal/array/reference2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_REFERENCE2_H -#define ZFP_REFERENCE2_H +#ifndef ZFP_REFERENCE2_HPP +#define ZFP_REFERENCE2_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/reference3.hpp b/include/zfp/internal/array/reference3.hpp index de58710ff..ecb52d30d 100644 --- a/include/zfp/internal/array/reference3.hpp +++ b/include/zfp/internal/array/reference3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_REFERENCE3_H -#define ZFP_REFERENCE3_H +#ifndef ZFP_REFERENCE3_HPP +#define ZFP_REFERENCE3_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/reference4.hpp b/include/zfp/internal/array/reference4.hpp index 231fff0a2..1d0c3ca3c 100644 --- a/include/zfp/internal/array/reference4.hpp +++ b/include/zfp/internal/array/reference4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_REFERENCE4_H -#define ZFP_REFERENCE4_H +#ifndef ZFP_REFERENCE4_HPP +#define ZFP_REFERENCE4_HPP namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store.hpp b/include/zfp/internal/array/store.hpp index 4b7b6b545..cd67e8726 100644 --- a/include/zfp/internal/array/store.hpp +++ b/include/zfp/internal/array/store.hpp @@ -1,9 +1,9 @@ -#ifndef ZFP_STORE_H -#define ZFP_STORE_H +#ifndef ZFP_STORE_HPP +#define ZFP_STORE_HPP #include #include -#include "zfp/memory.h" +#include "zfp/internal/array/memory.h" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store1.hpp b/include/zfp/internal/array/store1.hpp index 0505c20c5..cb401042e 100644 --- a/include/zfp/internal/array/store1.hpp +++ b/include/zfp/internal/array/store1.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_STORE1_H -#define ZFP_STORE1_H +#ifndef ZFP_STORE1_HPP +#define ZFP_STORE1_HPP -#include "zfp/store.h" +#include "zfp/internal/array/store.h" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store2.hpp b/include/zfp/internal/array/store2.hpp index 8dc60babc..8a44e0828 100644 --- a/include/zfp/internal/array/store2.hpp +++ b/include/zfp/internal/array/store2.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_STORE2_H -#define ZFP_STORE2_H +#ifndef ZFP_STORE2_HPP +#define ZFP_STORE2_HPP -#include "zfp/store.h" +#include "zfp/internal/array/store.h" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store3.hpp b/include/zfp/internal/array/store3.hpp index 7b26e196e..a2e279735 100644 --- a/include/zfp/internal/array/store3.hpp +++ b/include/zfp/internal/array/store3.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_STORE3_H -#define ZFP_STORE3_H +#ifndef ZFP_STORE3_HPP +#define ZFP_STORE3_HPP -#include "zfp/store.h" +#include "zfp/internal/array/store.h" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store4.hpp b/include/zfp/internal/array/store4.hpp index 75d404cfa..b0f5302c7 100644 --- a/include/zfp/internal/array/store4.hpp +++ b/include/zfp/internal/array/store4.hpp @@ -1,7 +1,7 @@ -#ifndef ZFP_STORE4_H -#define ZFP_STORE4_H +#ifndef ZFP_STORE4_HPP +#define ZFP_STORE4_HPP -#include "zfp/store.h" +#include "zfp/internal/array/store.h" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/traits.hpp b/include/zfp/internal/array/traits.hpp index 78f967807..d57ca8c35 100644 --- a/include/zfp/internal/array/traits.hpp +++ b/include/zfp/internal/array/traits.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_TRAITS_H -#define ZFP_TRAITS_H +#ifndef ZFP_TRAITS_HPP +#define ZFP_TRAITS_HPP namespace zfp { diff --git a/include/zfp/internal/array/view1.hpp b/include/zfp/internal/array/view1.hpp index 635f30077..a03acb0aa 100644 --- a/include/zfp/internal/array/view1.hpp +++ b/include/zfp/internal/array/view1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_VIEW1_H -#define ZFP_VIEW1_H +#ifndef ZFP_VIEW1_HPP +#define ZFP_VIEW1_HPP // 1D array views diff --git a/include/zfp/internal/array/view2.hpp b/include/zfp/internal/array/view2.hpp index 5ffddf62c..05d95be18 100644 --- a/include/zfp/internal/array/view2.hpp +++ b/include/zfp/internal/array/view2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_VIEW2_H -#define ZFP_VIEW2_H +#ifndef ZFP_VIEW2_HPP +#define ZFP_VIEW2_HPP // 2D array views diff --git a/include/zfp/internal/array/view3.hpp b/include/zfp/internal/array/view3.hpp index c41808a7e..1590560dc 100644 --- a/include/zfp/internal/array/view3.hpp +++ b/include/zfp/internal/array/view3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_VIEW3_H -#define ZFP_VIEW3_H +#ifndef ZFP_VIEW3_HPP +#define ZFP_VIEW3_HPP // 3D array views diff --git a/include/zfp/internal/array/view4.hpp b/include/zfp/internal/array/view4.hpp index 808e1e605..817e899c6 100644 --- a/include/zfp/internal/array/view4.hpp +++ b/include/zfp/internal/array/view4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_VIEW4_H -#define ZFP_VIEW4_H +#ifndef ZFP_VIEW4_HPP +#define ZFP_VIEW4_HPP // 4D array views From 4aafbc81ab51239a4abd8e3936e9e112dfb899b1 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Jul 2022 12:19:09 -0700 Subject: [PATCH 069/277] Update hpp include guards --- include/zfp.hpp | 4 ++-- include/zfp/array.hpp | 4 ++-- include/zfp/array1.hpp | 4 ++-- include/zfp/array2.hpp | 4 ++-- include/zfp/array3.hpp | 4 ++-- include/zfp/array4.hpp | 4 ++-- include/zfp/codec/generic.hpp | 4 ++-- include/zfp/codec/zfp.hpp | 4 ++-- include/zfp/constarray1.hpp | 4 ++-- include/zfp/constarray2.hpp | 4 ++-- include/zfp/constarray3.hpp | 4 ++-- include/zfp/constarray4.hpp | 4 ++-- include/zfp/factory.hpp | 6 +++--- include/zfp/index.hpp | 4 ++-- 14 files changed, 29 insertions(+), 29 deletions(-) diff --git a/include/zfp.hpp b/include/zfp.hpp index 9fa4fb895..b215ac0de 100644 --- a/include/zfp.hpp +++ b/include/zfp.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CPP_H -#define ZFP_CPP_H +#ifndef ZFP_HPP +#define ZFP_HPP #include "zfp.h" diff --git a/include/zfp/array.hpp b/include/zfp/array.hpp index 113c61f9b..07d5b08b6 100644 --- a/include/zfp/array.hpp +++ b/include/zfp/array.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ARRAY_H -#define ZFP_ARRAY_H +#ifndef ZFP_ARRAY_HPP +#define ZFP_ARRAY_HPP #include #include diff --git a/include/zfp/array1.hpp b/include/zfp/array1.hpp index 437f96997..e577bf8ea 100644 --- a/include/zfp/array1.hpp +++ b/include/zfp/array1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ARRAY1_H -#define ZFP_ARRAY1_H +#ifndef ZFP_ARRAY1_HPP +#define ZFP_ARRAY1_HPP #include #include diff --git a/include/zfp/array2.hpp b/include/zfp/array2.hpp index 439a05169..e008f8a14 100644 --- a/include/zfp/array2.hpp +++ b/include/zfp/array2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ARRAY2_H -#define ZFP_ARRAY2_H +#ifndef ZFP_ARRAY2_HPP +#define ZFP_ARRAY2_HPP #include #include diff --git a/include/zfp/array3.hpp b/include/zfp/array3.hpp index ee25d95b5..d91d64ba3 100644 --- a/include/zfp/array3.hpp +++ b/include/zfp/array3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ARRAY3_H -#define ZFP_ARRAY3_H +#ifndef ZFP_ARRAY3_HPP +#define ZFP_ARRAY3_HPP #include #include diff --git a/include/zfp/array4.hpp b/include/zfp/array4.hpp index 4d95d1413..eff5d99c7 100644 --- a/include/zfp/array4.hpp +++ b/include/zfp/array4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ARRAY4_H -#define ZFP_ARRAY4_H +#ifndef ZFP_ARRAY4_HPP +#define ZFP_ARRAY4_HPP #include #include diff --git a/include/zfp/codec/generic.hpp b/include/zfp/codec/generic.hpp index 4363b899a..0231592c0 100644 --- a/include/zfp/codec/generic.hpp +++ b/include/zfp/codec/generic.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_GENERIC_CODEC_H -#define ZFP_GENERIC_CODEC_H +#ifndef ZFP_GENERIC_CODEC_HPP +#define ZFP_GENERIC_CODEC_HPP // This CODEC allows interfacing with the zfp::array classes via a user-facing // scalar type, ExternalType (e.g., double), while storing data in memory using diff --git a/include/zfp/codec/zfp.hpp b/include/zfp/codec/zfp.hpp index d04067d9d..3b0a92c65 100644 --- a/include/zfp/codec/zfp.hpp +++ b/include/zfp/codec/zfp.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_ZFP_CODEC_H -#define ZFP_ZFP_CODEC_H +#ifndef ZFP_ZFP_CODEC_HPP +#define ZFP_ZFP_CODEC_HPP #include #include diff --git a/include/zfp/constarray1.hpp b/include/zfp/constarray1.hpp index 6081b17f4..d3b54db6b 100644 --- a/include/zfp/constarray1.hpp +++ b/include/zfp/constarray1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY1_H -#define ZFP_CARRAY1_H +#ifndef ZFP_CARRAY1_HPP +#define ZFP_CARRAY1_HPP #include #include diff --git a/include/zfp/constarray2.hpp b/include/zfp/constarray2.hpp index 81a7eacb4..459b70c4b 100644 --- a/include/zfp/constarray2.hpp +++ b/include/zfp/constarray2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY2_H -#define ZFP_CARRAY2_H +#ifndef ZFP_CARRAY2_HPP +#define ZFP_CARRAY2_HPP #include #include diff --git a/include/zfp/constarray3.hpp b/include/zfp/constarray3.hpp index 96ebd7fc9..5f5c80809 100644 --- a/include/zfp/constarray3.hpp +++ b/include/zfp/constarray3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY3_H -#define ZFP_CARRAY3_H +#ifndef ZFP_CARRAY3_HPP +#define ZFP_CARRAY3_HPP #include #include diff --git a/include/zfp/constarray4.hpp b/include/zfp/constarray4.hpp index 2b0159ed7..3af4a7725 100644 --- a/include/zfp/constarray4.hpp +++ b/include/zfp/constarray4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY4_H -#define ZFP_CARRAY4_H +#ifndef ZFP_CARRAY4_HPP +#define ZFP_CARRAY4_HPP #include #include diff --git a/include/zfp/factory.hpp b/include/zfp/factory.hpp index 5cbdaf82d..c7953541b 100644 --- a/include/zfp/factory.hpp +++ b/include/zfp/factory.hpp @@ -1,8 +1,8 @@ -#ifndef ZFP_FACTORY_H -#define ZFP_FACTORY_H +#ifndef ZFP_FACTORY_HPP +#define ZFP_FACTORY_HPP // ensure zfp/array.h has already been included -#ifndef ZFP_ARRAY_H +#ifndef ZFP_ARRAY_HPP #error "zfp/array.hpp must be included before zfp/factory.hpp" #endif diff --git a/include/zfp/index.hpp b/include/zfp/index.hpp index 285cb027b..4709760f0 100644 --- a/include/zfp/index.hpp +++ b/include/zfp/index.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_INDEX_H -#define ZFP_INDEX_H +#ifndef ZFP_INDEX_HPP +#define ZFP_INDEX_HPP #include From b2366c0e7ead232aa1b2edc44740ec3d8a2e01a7 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Jul 2022 14:12:03 -0700 Subject: [PATCH 070/277] updated inline src and moved cfp src --- cfp/CMakeLists.txt | 40 ++++++++++++++++++- cfp/{src => }/Makefile | 0 cfp/{src => }/cfparray.cpp | 0 cfp/{src => }/cfparray1d.cpp | 0 cfp/{src => }/cfparray1f.cpp | 0 cfp/{src => }/cfparray2d.cpp | 0 cfp/{src => }/cfparray2f.cpp | 0 cfp/{src => }/cfparray3d.cpp | 0 cfp/{src => }/cfparray3f.cpp | 0 cfp/{src => }/cfparray4d.cpp | 0 cfp/{src => }/cfparray4f.cpp | 0 cfp/{src => }/cfpheader.cpp | 0 cfp/src/CMakeLists.txt | 39 ------------------ cfp/{src => }/template/cfparray.cpp | 0 cfp/{src => }/template/cfparray1.cpp | 0 cfp/{src => }/template/cfparray2.cpp | 0 cfp/{src => }/template/cfparray3.cpp | 0 cfp/{src => }/template/cfparray4.cpp | 0 cfp/{src => }/template/cfpheader.cpp | 0 .../bitstream.c => include/zfp/bitstream.inl | 2 +- .../zfp/internal/zfp}/inline.h | 0 src/bitstream.c | 4 +- src/decode1d.c | 6 +-- src/decode1f.c | 6 +-- src/decode1i.c | 6 +-- src/decode1l.c | 6 +-- src/decode2d.c | 6 +-- src/decode2f.c | 6 +-- src/decode2i.c | 6 +-- src/decode2l.c | 6 +-- src/decode3d.c | 6 +-- src/decode3f.c | 6 +-- src/decode3i.c | 6 +-- src/decode3l.c | 6 +-- src/decode4d.c | 6 +-- src/decode4f.c | 6 +-- src/decode4i.c | 6 +-- src/decode4l.c | 6 +-- src/encode1d.c | 6 +-- src/encode1f.c | 6 +-- src/encode1i.c | 6 +-- src/encode1l.c | 6 +-- src/encode2d.c | 6 +-- src/encode2f.c | 6 +-- src/encode2i.c | 6 +-- src/encode2l.c | 6 +-- src/encode3d.c | 6 +-- src/encode3f.c | 6 +-- src/encode3i.c | 6 +-- src/encode3l.c | 6 +-- src/encode4d.c | 6 +-- src/encode4f.c | 6 +-- src/encode4i.c | 6 +-- src/encode4l.c | 6 +-- src/zfp.c | 2 +- 55 files changed, 139 insertions(+), 140 deletions(-) rename cfp/{src => }/Makefile (100%) rename cfp/{src => }/cfparray.cpp (100%) rename cfp/{src => }/cfparray1d.cpp (100%) rename cfp/{src => }/cfparray1f.cpp (100%) rename cfp/{src => }/cfparray2d.cpp (100%) rename cfp/{src => }/cfparray2f.cpp (100%) rename cfp/{src => }/cfparray3d.cpp (100%) rename cfp/{src => }/cfparray3f.cpp (100%) rename cfp/{src => }/cfparray4d.cpp (100%) rename cfp/{src => }/cfparray4f.cpp (100%) rename cfp/{src => }/cfpheader.cpp (100%) delete mode 100644 cfp/src/CMakeLists.txt rename cfp/{src => }/template/cfparray.cpp (100%) rename cfp/{src => }/template/cfparray1.cpp (100%) rename cfp/{src => }/template/cfparray2.cpp (100%) rename cfp/{src => }/template/cfparray3.cpp (100%) rename cfp/{src => }/template/cfparray4.cpp (100%) rename cfp/{src => }/template/cfpheader.cpp (100%) rename src/inline/bitstream.c => include/zfp/bitstream.inl (99%) rename {src/inline => include/zfp/internal/zfp}/inline.h (100%) diff --git a/cfp/CMakeLists.txt b/cfp/CMakeLists.txt index febd4f0ab..922710133 100644 --- a/cfp/CMakeLists.txt +++ b/cfp/CMakeLists.txt @@ -1 +1,39 @@ -add_subdirectory(src) +add_library(cfp cfparray.cpp) + +if(DEFINED CFP_NAMESPACE) + list(APPEND cfp_public_defs "CFP_NAMESPACE=${CFP_NAMESPACE}") +endif() + +list(APPEND cfp_private_defs ${zfp_compressed_array_defs}) + +if(WIN32 AND BUILD_SHARED_LIBS) + # define ZFP_SOURCE when compiling libcfp to export symbols to Windows DLL + list(APPEND cfp_public_defs ZFP_SHARED_LIBS) + list(APPEND cfp_private_defs ZFP_SOURCE) +endif() + +target_compile_definitions(cfp + PUBLIC ${cfp_public_defs} + PRIVATE ${cfp_private_defs}) + +target_include_directories(cfp + PUBLIC + $ + $ + $ + $ + PRIVATE + ${ZFP_SOURCE_DIR}/array + ${ZFP_SOURCE_DIR}/src +) + +target_link_libraries(cfp zfp) + +set_property(TARGET cfp PROPERTY VERSION ${ZFP_VERSION}) +set_property(TARGET cfp PROPERTY SOVERSION ${ZFP_VERSION_MAJOR}) +set_property(TARGET cfp PROPERTY OUTPUT_NAME ${ZFP_LIBRARY_PREFIX}cfp) + +install(TARGETS cfp EXPORT cfp-targets + RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} + LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} + ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) diff --git a/cfp/src/Makefile b/cfp/Makefile similarity index 100% rename from cfp/src/Makefile rename to cfp/Makefile diff --git a/cfp/src/cfparray.cpp b/cfp/cfparray.cpp similarity index 100% rename from cfp/src/cfparray.cpp rename to cfp/cfparray.cpp diff --git a/cfp/src/cfparray1d.cpp b/cfp/cfparray1d.cpp similarity index 100% rename from cfp/src/cfparray1d.cpp rename to cfp/cfparray1d.cpp diff --git a/cfp/src/cfparray1f.cpp b/cfp/cfparray1f.cpp similarity index 100% rename from cfp/src/cfparray1f.cpp rename to cfp/cfparray1f.cpp diff --git a/cfp/src/cfparray2d.cpp b/cfp/cfparray2d.cpp similarity index 100% rename from cfp/src/cfparray2d.cpp rename to cfp/cfparray2d.cpp diff --git a/cfp/src/cfparray2f.cpp b/cfp/cfparray2f.cpp similarity index 100% rename from cfp/src/cfparray2f.cpp rename to cfp/cfparray2f.cpp diff --git a/cfp/src/cfparray3d.cpp b/cfp/cfparray3d.cpp similarity index 100% rename from cfp/src/cfparray3d.cpp rename to cfp/cfparray3d.cpp diff --git a/cfp/src/cfparray3f.cpp b/cfp/cfparray3f.cpp similarity index 100% rename from cfp/src/cfparray3f.cpp rename to cfp/cfparray3f.cpp diff --git a/cfp/src/cfparray4d.cpp b/cfp/cfparray4d.cpp similarity index 100% rename from cfp/src/cfparray4d.cpp rename to cfp/cfparray4d.cpp diff --git a/cfp/src/cfparray4f.cpp b/cfp/cfparray4f.cpp similarity index 100% rename from cfp/src/cfparray4f.cpp rename to cfp/cfparray4f.cpp diff --git a/cfp/src/cfpheader.cpp b/cfp/cfpheader.cpp similarity index 100% rename from cfp/src/cfpheader.cpp rename to cfp/cfpheader.cpp diff --git a/cfp/src/CMakeLists.txt b/cfp/src/CMakeLists.txt deleted file mode 100644 index 922710133..000000000 --- a/cfp/src/CMakeLists.txt +++ /dev/null @@ -1,39 +0,0 @@ -add_library(cfp cfparray.cpp) - -if(DEFINED CFP_NAMESPACE) - list(APPEND cfp_public_defs "CFP_NAMESPACE=${CFP_NAMESPACE}") -endif() - -list(APPEND cfp_private_defs ${zfp_compressed_array_defs}) - -if(WIN32 AND BUILD_SHARED_LIBS) - # define ZFP_SOURCE when compiling libcfp to export symbols to Windows DLL - list(APPEND cfp_public_defs ZFP_SHARED_LIBS) - list(APPEND cfp_private_defs ZFP_SOURCE) -endif() - -target_compile_definitions(cfp - PUBLIC ${cfp_public_defs} - PRIVATE ${cfp_private_defs}) - -target_include_directories(cfp - PUBLIC - $ - $ - $ - $ - PRIVATE - ${ZFP_SOURCE_DIR}/array - ${ZFP_SOURCE_DIR}/src -) - -target_link_libraries(cfp zfp) - -set_property(TARGET cfp PROPERTY VERSION ${ZFP_VERSION}) -set_property(TARGET cfp PROPERTY SOVERSION ${ZFP_VERSION_MAJOR}) -set_property(TARGET cfp PROPERTY OUTPUT_NAME ${ZFP_LIBRARY_PREFIX}cfp) - -install(TARGETS cfp EXPORT cfp-targets - RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} - LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} - ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) diff --git a/cfp/src/template/cfparray.cpp b/cfp/template/cfparray.cpp similarity index 100% rename from cfp/src/template/cfparray.cpp rename to cfp/template/cfparray.cpp diff --git a/cfp/src/template/cfparray1.cpp b/cfp/template/cfparray1.cpp similarity index 100% rename from cfp/src/template/cfparray1.cpp rename to cfp/template/cfparray1.cpp diff --git a/cfp/src/template/cfparray2.cpp b/cfp/template/cfparray2.cpp similarity index 100% rename from cfp/src/template/cfparray2.cpp rename to cfp/template/cfparray2.cpp diff --git a/cfp/src/template/cfparray3.cpp b/cfp/template/cfparray3.cpp similarity index 100% rename from cfp/src/template/cfparray3.cpp rename to cfp/template/cfparray3.cpp diff --git a/cfp/src/template/cfparray4.cpp b/cfp/template/cfparray4.cpp similarity index 100% rename from cfp/src/template/cfparray4.cpp rename to cfp/template/cfparray4.cpp diff --git a/cfp/src/template/cfpheader.cpp b/cfp/template/cfpheader.cpp similarity index 100% rename from cfp/src/template/cfpheader.cpp rename to cfp/template/cfpheader.cpp diff --git a/src/inline/bitstream.c b/include/zfp/bitstream.inl similarity index 99% rename from src/inline/bitstream.c rename to include/zfp/bitstream.inl index a4a60f477..987f5a42a 100644 --- a/src/inline/bitstream.c +++ b/include/zfp/bitstream.inl @@ -112,7 +112,7 @@ The following assumptions and restrictions apply: #define inline_ #endif -#include "bitstream.h" +#include "zfp/bitstream.h" /* satisfy compiler when args unused */ #define unused_(x) ((void)(x)) diff --git a/src/inline/inline.h b/include/zfp/internal/zfp/inline.h similarity index 100% rename from src/inline/inline.h rename to include/zfp/internal/zfp/inline.h diff --git a/src/bitstream.c b/src/bitstream.c index 3a7476673..29a4543a9 100644 --- a/src/bitstream.c +++ b/src/bitstream.c @@ -1,4 +1,4 @@ -#include "bitstream.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.h" +#include "zfp/bitstream.inl" const size_t stream_word_bits = wsize; diff --git a/src/decode1d.c b/src/decode1d.c index b5bbd6693..b95995fa7 100644 --- a/src/decode1d.c +++ b/src/decode1d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec1.c" diff --git a/src/decode1f.c b/src/decode1f.c index 83085a861..f08119f7f 100644 --- a/src/decode1f.c +++ b/src/decode1f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec1.c" diff --git a/src/decode1i.c b/src/decode1i.c index b0e61ab79..b148641eb 100644 --- a/src/decode1i.c +++ b/src/decode1i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec1.c" #include "template/decode.c" diff --git a/src/decode1l.c b/src/decode1l.c index a063a1fbd..d79e8e46b 100644 --- a/src/decode1l.c +++ b/src/decode1l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec1.c" #include "template/decode.c" diff --git a/src/decode2d.c b/src/decode2d.c index ced3d5d76..d7f3a77c6 100644 --- a/src/decode2d.c +++ b/src/decode2d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec2.c" diff --git a/src/decode2f.c b/src/decode2f.c index f6d823a46..5d44e0725 100644 --- a/src/decode2f.c +++ b/src/decode2f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec2.c" diff --git a/src/decode2i.c b/src/decode2i.c index e4b5235f2..579eaa82f 100644 --- a/src/decode2i.c +++ b/src/decode2i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec2.c" #include "template/decode.c" diff --git a/src/decode2l.c b/src/decode2l.c index 80031eaff..b4d871f5c 100644 --- a/src/decode2l.c +++ b/src/decode2l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec2.c" #include "template/decode.c" diff --git a/src/decode3d.c b/src/decode3d.c index 823c64fdf..e9291aa48 100644 --- a/src/decode3d.c +++ b/src/decode3d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec3.c" diff --git a/src/decode3f.c b/src/decode3f.c index 2e2724ff9..cc517b138 100644 --- a/src/decode3f.c +++ b/src/decode3f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec3.c" diff --git a/src/decode3i.c b/src/decode3i.c index 6888f245b..0eb05deaf 100644 --- a/src/decode3i.c +++ b/src/decode3i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec3.c" #include "template/decode.c" diff --git a/src/decode3l.c b/src/decode3l.c index 225b6bc75..d895d0e7a 100644 --- a/src/decode3l.c +++ b/src/decode3l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec3.c" #include "template/decode.c" diff --git a/src/decode4d.c b/src/decode4d.c index 8a047b629..38861b5d2 100644 --- a/src/decode4d.c +++ b/src/decode4d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec4.c" diff --git a/src/decode4f.c b/src/decode4f.c index 6ec6bf06c..7ef87f10d 100644 --- a/src/decode4f.c +++ b/src/decode4f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec4.c" diff --git a/src/decode4i.c b/src/decode4i.c index 309eee238..ade99493f 100644 --- a/src/decode4i.c +++ b/src/decode4i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec4.c" #include "template/decode.c" diff --git a/src/decode4l.c b/src/decode4l.c index c07c85cce..bbbdefbbd 100644 --- a/src/decode4l.c +++ b/src/decode4l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec4.c" #include "template/decode.c" diff --git a/src/encode1d.c b/src/encode1d.c index 7f8f3ca1a..43f5101c1 100644 --- a/src/encode1d.c +++ b/src/encode1d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec1.c" diff --git a/src/encode1f.c b/src/encode1f.c index 5fe4812cc..ae509d534 100644 --- a/src/encode1f.c +++ b/src/encode1f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec1.c" diff --git a/src/encode1i.c b/src/encode1i.c index f3069de93..ea3593cdd 100644 --- a/src/encode1i.c +++ b/src/encode1i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec1.c" #include "template/encode.c" diff --git a/src/encode1l.c b/src/encode1l.c index 62a1814af..e9415e209 100644 --- a/src/encode1l.c +++ b/src/encode1l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block1.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec1.c" #include "template/encode.c" diff --git a/src/encode2d.c b/src/encode2d.c index 4dba8923c..8f4458923 100644 --- a/src/encode2d.c +++ b/src/encode2d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec2.c" diff --git a/src/encode2f.c b/src/encode2f.c index d667ff7ae..814a18a2d 100644 --- a/src/encode2f.c +++ b/src/encode2f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec2.c" diff --git a/src/encode2i.c b/src/encode2i.c index 264c44907..8417031f0 100644 --- a/src/encode2i.c +++ b/src/encode2i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec2.c" #include "template/encode.c" diff --git a/src/encode2l.c b/src/encode2l.c index fdea8a018..87f5a2f34 100644 --- a/src/encode2l.c +++ b/src/encode2l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block2.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec2.c" #include "template/encode.c" diff --git a/src/encode3d.c b/src/encode3d.c index 548542138..55f55d624 100644 --- a/src/encode3d.c +++ b/src/encode3d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec3.c" diff --git a/src/encode3f.c b/src/encode3f.c index 2e9db3f27..de3bbaf40 100644 --- a/src/encode3f.c +++ b/src/encode3f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec3.c" diff --git a/src/encode3i.c b/src/encode3i.c index 6dcfbe53e..257a1ecd2 100644 --- a/src/encode3i.c +++ b/src/encode3i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec3.c" #include "template/encode.c" diff --git a/src/encode3l.c b/src/encode3l.c index 26c9077e9..c62696997 100644 --- a/src/encode3l.c +++ b/src/encode3l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block3.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec3.c" #include "template/encode.c" diff --git a/src/encode4d.c b/src/encode4d.c index c96aae061..346f17471 100644 --- a/src/encode4d.c +++ b/src/encode4d.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsd.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec4.c" diff --git a/src/encode4f.c b/src/encode4f.c index 36cc28318..b855262b0 100644 --- a/src/encode4f.c +++ b/src/encode4f.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsf.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codecf.c" #include "template/codec4.c" diff --git a/src/encode4i.c b/src/encode4i.c index 86fc5ace2..5bed6cdf9 100644 --- a/src/encode4i.c +++ b/src/encode4i.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsi.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec4.c" #include "template/encode.c" diff --git a/src/encode4l.c b/src/encode4l.c index e8a382817..fd84e5a1c 100644 --- a/src/encode4l.c +++ b/src/encode4l.c @@ -1,11 +1,11 @@ -#include "inline/inline.h" +#include "zfp/internal/zfp/inline.h" #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "block4.h" #include "traitsl.h" #include "template/template.h" #include "template/codec.h" -#include "inline/bitstream.c" +#include "zfp/bitstream.inl" #include "template/codec.c" #include "template/codec4.c" #include "template/encode.c" diff --git a/src/zfp.c b/src/zfp.c index 81625ad7b..e8c75a871 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -3,7 +3,7 @@ #include #include #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "zfp/version.h" #include "template/template.h" From c7326188710e970b9c296559cddf2d040d33a930 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 6 Jul 2022 14:19:42 -0700 Subject: [PATCH 071/277] Update cmake --- cfp/CMakeLists.txt | 5 +---- cfp/{cfparray.cpp => cfp.cpp} | 2 +- src/CMakeLists.txt | 3 +-- 3 files changed, 3 insertions(+), 7 deletions(-) rename cfp/{cfparray.cpp => cfp.cpp} (99%) diff --git a/cfp/CMakeLists.txt b/cfp/CMakeLists.txt index 922710133..3d8af6ecf 100644 --- a/cfp/CMakeLists.txt +++ b/cfp/CMakeLists.txt @@ -1,4 +1,4 @@ -add_library(cfp cfparray.cpp) +add_library(cfp cfp.cpp) if(DEFINED CFP_NAMESPACE) list(APPEND cfp_public_defs "CFP_NAMESPACE=${CFP_NAMESPACE}") @@ -19,11 +19,8 @@ target_compile_definitions(cfp target_include_directories(cfp PUBLIC $ - $ - $ $ PRIVATE - ${ZFP_SOURCE_DIR}/array ${ZFP_SOURCE_DIR}/src ) diff --git a/cfp/cfparray.cpp b/cfp/cfp.cpp similarity index 99% rename from cfp/cfparray.cpp rename to cfp/cfp.cpp index 358f480d7..4cbb14d8f 100644 --- a/cfp/cfparray.cpp +++ b/cfp/cfp.cpp @@ -1,5 +1,5 @@ #include "cfpheader.cpp" -#include "cfparray.h" +#include "zfp/cfp.h" #include "cfparray1f.cpp" #include "cfparray1d.cpp" diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index c0f63965c..fd5702e58 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -54,8 +54,7 @@ target_include_directories(zfp PUBLIC $ $ - INTERFACE - $) +) set_property(TARGET zfp PROPERTY VERSION ${ZFP_VERSION}) set_property(TARGET zfp PROPERTY SOVERSION ${ZFP_VERSION_MAJOR}) From eeac3a28a2d9d7e465e41ecca302d32752c8269b Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 7 Jul 2022 09:13:05 -0700 Subject: [PATCH 072/277] Updated cfp includes, fixed build issues --- CMakeLists.txt | 8 +++--- cfp/cfparray1d.cpp | 4 +-- cfp/cfparray1f.cpp | 4 +-- cfp/cfparray2d.cpp | 4 +-- cfp/cfparray2f.cpp | 4 +-- cfp/cfparray3d.cpp | 4 +-- cfp/cfparray3f.cpp | 4 +-- cfp/cfparray4d.cpp | 4 +-- cfp/cfparray4f.cpp | 4 +-- cfp/cfpheader.cpp | 28 ++++++++++----------- include/zfp/array1.hpp | 2 +- include/zfp/array2.hpp | 2 +- include/zfp/array3.hpp | 2 +- include/zfp/array4.hpp | 2 +- include/zfp/codec/{zfp.hpp => zfpcodec.hpp} | 6 ++--- include/zfp/constarray1.hpp | 2 +- include/zfp/constarray2.hpp | 2 +- include/zfp/constarray3.hpp | 2 +- include/zfp/constarray4.hpp | 2 +- include/zfp/index.hpp | 1 + include/zfp/internal/array/store.hpp | 2 +- include/zfp/internal/array/store1.hpp | 2 +- include/zfp/internal/array/store2.hpp | 2 +- include/zfp/internal/array/store3.hpp | 2 +- include/zfp/internal/array/store4.hpp | 2 +- 25 files changed, 51 insertions(+), 50 deletions(-) rename include/zfp/codec/{zfp.hpp => zfpcodec.hpp} (98%) diff --git a/CMakeLists.txt b/CMakeLists.txt index ca5df5eb7..2d0615aca 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -326,11 +326,11 @@ endif() #------------------------------------------------------------------------------# # Header install #------------------------------------------------------------------------------# -install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) -install(DIRECTORY array/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) - if(BUILD_CFP) - install(DIRECTORY cfp/include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) + install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) +else() + install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} + PATTERN "cfp" EXCLUDE) endif() #------------------------------------------------------------------------------# # Build type: one of None, Debug, Release, RelWithDebInfo, MinSizeRel diff --git a/cfp/cfparray1d.cpp b/cfp/cfparray1d.cpp index 35e18ace2..3a76b65f9 100644 --- a/cfp/cfparray1d.cpp +++ b/cfp/cfparray1d.cpp @@ -1,5 +1,5 @@ -#include "cfparray1d.h" -#include "zfparray1.h" +#include "zfp/internal/cfp/array1d.h" +#include "zfp/array1.hpp" #include "template/template.h" diff --git a/cfp/cfparray1f.cpp b/cfp/cfparray1f.cpp index dd6859bb1..2df705301 100644 --- a/cfp/cfparray1f.cpp +++ b/cfp/cfparray1f.cpp @@ -1,5 +1,5 @@ -#include "cfparray1f.h" -#include "zfparray1.h" +#include "zfp/internal/cfp/array1f.h" +#include "zfp/array1.hpp" #include "template/template.h" diff --git a/cfp/cfparray2d.cpp b/cfp/cfparray2d.cpp index f35acd2c8..fa3051b12 100644 --- a/cfp/cfparray2d.cpp +++ b/cfp/cfparray2d.cpp @@ -1,5 +1,5 @@ -#include "cfparray2d.h" -#include "zfparray2.h" +#include "zfp/internal/cfp/array2d.h" +#include "zfp/array2.hpp" #include "template/template.h" diff --git a/cfp/cfparray2f.cpp b/cfp/cfparray2f.cpp index 893710c7d..ebfd1d9db 100644 --- a/cfp/cfparray2f.cpp +++ b/cfp/cfparray2f.cpp @@ -1,5 +1,5 @@ -#include "cfparray2f.h" -#include "zfparray2.h" +#include "zfp/internal/cfp/array2f.h" +#include "zfp/array2.hpp" #include "template/template.h" diff --git a/cfp/cfparray3d.cpp b/cfp/cfparray3d.cpp index aea61b0aa..100d639a0 100644 --- a/cfp/cfparray3d.cpp +++ b/cfp/cfparray3d.cpp @@ -1,5 +1,5 @@ -#include "cfparray3d.h" -#include "zfparray3.h" +#include "zfp/internal/cfp/array3d.h" +#include "zfp/array3.hpp" #include "template/template.h" diff --git a/cfp/cfparray3f.cpp b/cfp/cfparray3f.cpp index f3220f656..b5cafb71b 100644 --- a/cfp/cfparray3f.cpp +++ b/cfp/cfparray3f.cpp @@ -1,5 +1,5 @@ -#include "cfparray3f.h" -#include "zfparray3.h" +#include "zfp/internal/cfp/array3f.h" +#include "zfp/array3.hpp" #include "template/template.h" diff --git a/cfp/cfparray4d.cpp b/cfp/cfparray4d.cpp index 0854fa85b..bf1a2b06d 100644 --- a/cfp/cfparray4d.cpp +++ b/cfp/cfparray4d.cpp @@ -1,5 +1,5 @@ -#include "cfparray4d.h" -#include "zfparray4.h" +#include "zfp/internal/cfp/array4d.h" +#include "zfp/array4.hpp" #include "template/template.h" diff --git a/cfp/cfparray4f.cpp b/cfp/cfparray4f.cpp index 4e52600f2..ca6bf0ddd 100644 --- a/cfp/cfparray4f.cpp +++ b/cfp/cfparray4f.cpp @@ -1,5 +1,5 @@ -#include "cfparray4f.h" -#include "zfparray4.h" +#include "zfp/internal/cfp/array4f.h" +#include "zfp/array4.hpp" #include "template/template.h" diff --git a/cfp/cfpheader.cpp b/cfp/cfpheader.cpp index 667e63852..efadd8677 100644 --- a/cfp/cfpheader.cpp +++ b/cfp/cfpheader.cpp @@ -1,17 +1,17 @@ -#include "zfparray1.h" -#include "zfparray2.h" -#include "zfparray3.h" -#include "zfparray4.h" -#include "zfp/zfpheader.h" -#include "cfpheader.h" -#include "cfparray1f.h" -#include "cfparray1d.h" -#include "cfparray2f.h" -#include "cfparray2d.h" -#include "cfparray3f.h" -#include "cfparray3d.h" -#include "cfparray4f.h" -#include "cfparray4d.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/array4.hpp" +#include "zfp/internal/array/zfpheader.hpp" +#include "zfp/internal/cfp/header.h" +#include "zfp/internal/cfp/array1f.h" +#include "zfp/internal/cfp/array1d.h" +#include "zfp/internal/cfp/array2f.h" +#include "zfp/internal/cfp/array2d.h" +#include "zfp/internal/cfp/array3f.h" +#include "zfp/internal/cfp/array3d.h" +#include "zfp/internal/cfp/array4f.h" +#include "zfp/internal/cfp/array4d.h" #include "template/template.h" diff --git a/include/zfp/array1.hpp b/include/zfp/array1.hpp index e577bf8ea..0a01fc4d9 100644 --- a/include/zfp/array1.hpp +++ b/include/zfp/array1.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache1.hpp" #include "zfp/internal/array/handle1.hpp" #include "zfp/internal/array/iterator1.hpp" diff --git a/include/zfp/array2.hpp b/include/zfp/array2.hpp index e008f8a14..3ffdb94e9 100644 --- a/include/zfp/array2.hpp +++ b/include/zfp/array2.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache2.hpp" #include "zfp/internal/array/handle2.hpp" #include "zfp/internal/array/iterator2.hpp" diff --git a/include/zfp/array3.hpp b/include/zfp/array3.hpp index d91d64ba3..a735952cf 100644 --- a/include/zfp/array3.hpp +++ b/include/zfp/array3.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache3.hpp" #include "zfp/internal/array/handle3.hpp" #include "zfp/internal/array/iterator3.hpp" diff --git a/include/zfp/array4.hpp b/include/zfp/array4.hpp index eff5d99c7..63b86ff5e 100644 --- a/include/zfp/array4.hpp +++ b/include/zfp/array4.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache4.hpp" #include "zfp/internal/array/handle4.hpp" #include "zfp/internal/array/iterator4.hpp" diff --git a/include/zfp/codec/zfp.hpp b/include/zfp/codec/zfpcodec.hpp similarity index 98% rename from include/zfp/codec/zfp.hpp rename to include/zfp/codec/zfpcodec.hpp index 3b0a92c65..96399b710 100644 --- a/include/zfp/codec/zfp.hpp +++ b/include/zfp/codec/zfpcodec.hpp @@ -6,8 +6,8 @@ #include #include "zfp.h" #include "zfp.hpp" -#include "zfp/internal/array/memory.h" -#include "zfp/internal/array/traits.h" +#include "zfp/internal/array/memory.hpp" +#include "zfp/internal/array/traits.hpp" namespace zfp { namespace codec { @@ -115,7 +115,7 @@ class zfp_base { static const zfp_type type = zfp::trait::type; // scalar type // zfp::codec::zfp_base::header class for array (de)serialization - #include "zfp/zfpheader.h" + #include "zfp/internal/array/zfpheader.hpp" protected: // deep copy diff --git a/include/zfp/constarray1.hpp b/include/zfp/constarray1.hpp index d3b54db6b..6e339e679 100644 --- a/include/zfp/constarray1.hpp +++ b/include/zfp/constarray1.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache1.hpp" #include "zfp/internal/array/handle1.hpp" #include "zfp/internal/array/iterator1.hpp" diff --git a/include/zfp/constarray2.hpp b/include/zfp/constarray2.hpp index 459b70c4b..42335c4d6 100644 --- a/include/zfp/constarray2.hpp +++ b/include/zfp/constarray2.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache2.hpp" #include "zfp/internal/array/handle2.hpp" #include "zfp/internal/array/iterator2.hpp" diff --git a/include/zfp/constarray3.hpp b/include/zfp/constarray3.hpp index 5f5c80809..dbe28920d 100644 --- a/include/zfp/constarray3.hpp +++ b/include/zfp/constarray3.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/cache3.hpp" #include "zfp/handle3.hpp" #include "zfp/iterator3.hpp" diff --git a/include/zfp/constarray4.hpp b/include/zfp/constarray4.hpp index 3af4a7725..9846d1f79 100644 --- a/include/zfp/constarray4.hpp +++ b/include/zfp/constarray4.hpp @@ -6,7 +6,7 @@ #include #include "zfp/array.hpp" #include "zfp/index.hpp" -#include "zfp/codec/zfp.hpp" +#include "zfp/codec/zfpcodec.hpp" #include "zfp/internal/array/cache4.hpp" #include "zfp/internal/array/handle4.hpp" #include "zfp/internal/array/iterator4.hpp" diff --git a/include/zfp/index.hpp b/include/zfp/index.hpp index 4709760f0..6505af218 100644 --- a/include/zfp/index.hpp +++ b/include/zfp/index.hpp @@ -2,6 +2,7 @@ #define ZFP_INDEX_HPP #include +#include "zfp/internal/array/memory.hpp" namespace zfp { namespace index { diff --git a/include/zfp/internal/array/store.hpp b/include/zfp/internal/array/store.hpp index cd67e8726..f0590f9ee 100644 --- a/include/zfp/internal/array/store.hpp +++ b/include/zfp/internal/array/store.hpp @@ -3,7 +3,7 @@ #include #include -#include "zfp/internal/array/memory.h" +#include "zfp/internal/array/memory.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store1.hpp b/include/zfp/internal/array/store1.hpp index cb401042e..aeb05fa8e 100644 --- a/include/zfp/internal/array/store1.hpp +++ b/include/zfp/internal/array/store1.hpp @@ -1,7 +1,7 @@ #ifndef ZFP_STORE1_HPP #define ZFP_STORE1_HPP -#include "zfp/internal/array/store.h" +#include "zfp/internal/array/store.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store2.hpp b/include/zfp/internal/array/store2.hpp index 8a44e0828..466067ac4 100644 --- a/include/zfp/internal/array/store2.hpp +++ b/include/zfp/internal/array/store2.hpp @@ -1,7 +1,7 @@ #ifndef ZFP_STORE2_HPP #define ZFP_STORE2_HPP -#include "zfp/internal/array/store.h" +#include "zfp/internal/array/store.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store3.hpp b/include/zfp/internal/array/store3.hpp index a2e279735..cb2afb734 100644 --- a/include/zfp/internal/array/store3.hpp +++ b/include/zfp/internal/array/store3.hpp @@ -1,7 +1,7 @@ #ifndef ZFP_STORE3_HPP #define ZFP_STORE3_HPP -#include "zfp/internal/array/store.h" +#include "zfp/internal/array/store.hpp" namespace zfp { namespace internal { diff --git a/include/zfp/internal/array/store4.hpp b/include/zfp/internal/array/store4.hpp index b0f5302c7..dbea0c986 100644 --- a/include/zfp/internal/array/store4.hpp +++ b/include/zfp/internal/array/store4.hpp @@ -1,7 +1,7 @@ #ifndef ZFP_STORE4_HPP #define ZFP_STORE4_HPP -#include "zfp/internal/array/store.h" +#include "zfp/internal/array/store.hpp" namespace zfp { namespace internal { From 615e86f4be5abe5475e6c0c9387c52616a0948be Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 7 Jul 2022 09:20:42 -0700 Subject: [PATCH 073/277] Update headers in examples and fix bad include --- examples/array.cpp | 2 +- examples/diffusion.cpp | 6 +++--- examples/diffusionC.c | 2 +- examples/iterator.cpp | 6 +++--- examples/iteratorC.c | 2 +- include/zfp/codec/generic.hpp | 6 +++--- 6 files changed, 12 insertions(+), 12 deletions(-) diff --git a/examples/array.cpp b/examples/array.cpp index 546de33ce..233cb36ee 100644 --- a/examples/array.cpp +++ b/examples/array.cpp @@ -2,7 +2,7 @@ #include #include -#include "zfparray2.h" +#include "zfp/array2.hpp" int main() { diff --git a/examples/diffusion.cpp b/examples/diffusion.cpp index cd3fb362e..efe105002 100644 --- a/examples/diffusion.cpp +++ b/examples/diffusion.cpp @@ -6,9 +6,9 @@ #include #include #include -#include "zfparray2.h" -#include "zfpcarray2.h" -#include "gencodec.h" +#include "zfp/array2.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/codec/generic.hpp" #include "array2d.h" // add half precision if compiler supports it diff --git a/examples/diffusionC.c b/examples/diffusionC.c index a5d88a42a..fe90a4d33 100644 --- a/examples/diffusionC.c +++ b/examples/diffusionC.c @@ -7,7 +7,7 @@ forward Euler finite difference solution to the heat equation on a 2D grid #include #include -#include "cfparray.h" +#include "zfp/cfp.h" #define _ (CFP_NAMESPACE.array2d) #define MAX(x, y) (((nx) > (ny)) ? (nx) : (ny)) diff --git a/examples/iterator.cpp b/examples/iterator.cpp index 77bf1c5ff..94f907ded 100644 --- a/examples/iterator.cpp +++ b/examples/iterator.cpp @@ -1,9 +1,9 @@ #include #include #include -#include "zfparray1.h" -#include "zfparray2.h" -#include "zfparray3.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" void print1(zfp::array1::pointer p, size_t n) { diff --git a/examples/iteratorC.c b/examples/iteratorC.c index fcaf83456..a2ce139f7 100644 --- a/examples/iteratorC.c +++ b/examples/iteratorC.c @@ -1,6 +1,6 @@ #include #include -#include "cfparray.h" +#include "zfp/cfp.h" void print1(cfp_ptr1d p, size_t n) { diff --git a/include/zfp/codec/generic.hpp b/include/zfp/codec/generic.hpp index 0231592c0..55b398d8d 100644 --- a/include/zfp/codec/generic.hpp +++ b/include/zfp/codec/generic.hpp @@ -17,8 +17,8 @@ #include #include "zfp.h" #include "zfp/array.hpp" -#include "zfp/internal/array/memory.h" -#include "zfp/internal/array/traits.h" +#include "zfp/internal/array/memory.hpp" +#include "zfp/internal/array/traits.hpp" namespace zfp { namespace codec { @@ -144,7 +144,7 @@ class generic_base { static const zfp_type type = zfp::trait::type; // scalar type // zfp::codec::generic_base::header class for array (de)serialization - #include "zfp/genheader.h" + #include "zfp/internal/array/genheader.hpp" protected: // pointer to beginning of block From f56070f6a122e6c83dc54c816253e9ebfcf7ef02 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 7 Jul 2022 10:35:58 -0700 Subject: [PATCH 074/277] updated tests fixed additional missing includes --- include/zfp/constarray3.hpp | 14 +++++++------- include/zfp/factory.hpp | 8 ++++---- tests/array/array/testArray1d.cpp | 10 +++++----- tests/array/array/testArray1dIters.cpp | 2 +- tests/array/array/testArray1dPtrs.cpp | 2 +- tests/array/array/testArray1dRefs.cpp | 2 +- tests/array/array/testArray1dViewIters.cpp | 2 +- tests/array/array/testArray1dViewPtrs.cpp | 2 +- tests/array/array/testArray1dViews.cpp | 2 +- tests/array/array/testArray1f.cpp | 10 +++++----- tests/array/array/testArray1fIters.cpp | 2 +- tests/array/array/testArray1fPtrs.cpp | 2 +- tests/array/array/testArray1fRefs.cpp | 2 +- tests/array/array/testArray1fViewIters.cpp | 2 +- tests/array/array/testArray1fViewPtrs.cpp | 2 +- tests/array/array/testArray1fViews.cpp | 2 +- tests/array/array/testArray2d.cpp | 10 +++++----- tests/array/array/testArray2dIters.cpp | 2 +- tests/array/array/testArray2dPtrs.cpp | 2 +- tests/array/array/testArray2dRefs.cpp | 2 +- tests/array/array/testArray2dViewIters.cpp | 2 +- tests/array/array/testArray2dViewPtrs.cpp | 2 +- tests/array/array/testArray2dViews.cpp | 2 +- tests/array/array/testArray2f.cpp | 10 +++++----- tests/array/array/testArray2fIters.cpp | 2 +- tests/array/array/testArray2fPtrs.cpp | 2 +- tests/array/array/testArray2fRefs.cpp | 2 +- tests/array/array/testArray2fViewIters.cpp | 2 +- tests/array/array/testArray2fViewPtrs.cpp | 2 +- tests/array/array/testArray2fViews.cpp | 2 +- tests/array/array/testArray3d.cpp | 10 +++++----- tests/array/array/testArray3dIters.cpp | 2 +- tests/array/array/testArray3dPtrs.cpp | 2 +- tests/array/array/testArray3dRefs.cpp | 2 +- tests/array/array/testArray3dViewIters.cpp | 2 +- tests/array/array/testArray3dViewPtrs.cpp | 2 +- tests/array/array/testArray3dViews.cpp | 2 +- tests/array/array/testArray3f.cpp | 10 +++++----- tests/array/array/testArray3fIters.cpp | 2 +- tests/array/array/testArray3fPtrs.cpp | 2 +- tests/array/array/testArray3fRefs.cpp | 2 +- tests/array/array/testArray3fViewIters.cpp | 2 +- tests/array/array/testArray3fViewPtrs.cpp | 2 +- tests/array/array/testArray3fViews.cpp | 2 +- tests/array/array/testArray4d.cpp | 10 +++++----- tests/array/array/testArray4dIters.cpp | 2 +- tests/array/array/testArray4dPtrs.cpp | 2 +- tests/array/array/testArray4dRefs.cpp | 2 +- tests/array/array/testArray4dViewIters.cpp | 2 +- tests/array/array/testArray4dViewPtrs.cpp | 2 +- tests/array/array/testArray4dViews.cpp | 2 +- tests/array/array/testArray4f.cpp | 10 +++++----- tests/array/array/testArray4fIters.cpp | 2 +- tests/array/array/testArray4fPtrs.cpp | 2 +- tests/array/array/testArray4fRefs.cpp | 2 +- tests/array/array/testArray4fViewIters.cpp | 2 +- tests/array/array/testArray4fViewPtrs.cpp | 2 +- tests/array/array/testArray4fViews.cpp | 2 +- tests/array/array/testArrayBase.cpp | 2 +- tests/array/array/testConstruct.cpp | 8 ++++---- tests/array/constArray/testConstArray1d.cpp | 10 +++++----- tests/array/constArray/testConstArray1f.cpp | 10 +++++----- tests/array/constArray/testConstArray2d.cpp | 10 +++++----- tests/array/constArray/testConstArray2f.cpp | 10 +++++----- tests/array/constArray/testConstArray3d.cpp | 10 +++++----- tests/array/constArray/testConstArray3f.cpp | 10 +++++----- tests/array/constArray/testConstArray4d.cpp | 10 +++++----- tests/array/constArray/testConstArray4f.cpp | 10 +++++----- tests/array/decode/testTemplatedDecode1d.cpp | 2 +- tests/array/decode/testTemplatedDecode1f.cpp | 2 +- tests/array/decode/testTemplatedDecode2d.cpp | 2 +- tests/array/decode/testTemplatedDecode2f.cpp | 2 +- tests/array/decode/testTemplatedDecode3d.cpp | 2 +- tests/array/decode/testTemplatedDecode3f.cpp | 2 +- tests/array/decode/testTemplatedDecode4d.cpp | 2 +- tests/array/decode/testTemplatedDecode4f.cpp | 2 +- tests/array/encode/testTemplatedEncode1d.cpp | 2 +- tests/array/encode/testTemplatedEncode1f.cpp | 2 +- tests/array/encode/testTemplatedEncode2d.cpp | 2 +- tests/array/encode/testTemplatedEncode2f.cpp | 2 +- tests/array/encode/testTemplatedEncode3d.cpp | 2 +- tests/array/encode/testTemplatedEncode3f.cpp | 2 +- tests/array/encode/testTemplatedEncode4d.cpp | 2 +- tests/array/encode/testTemplatedEncode4f.cpp | 2 +- tests/array/utils/predicates.h | 2 +- tests/array/zfp/testAlignedMemory.cpp | 2 +- tests/cfp/testCfpArray_source.c | 2 +- tests/cfp/testCfpNamespace.c | 2 +- tests/src/execPolicy/testOmpInternal.c | 2 +- tests/src/inline/testBitstream.c | 4 ++-- tests/src/inline/testBitstreamSmallWsize.c | 4 ++-- tests/src/inline/testBitstreamStrided.c | 4 ++-- tests/testviews.cpp | 4 ++-- tests/testzfp.cpp | 8 ++++---- tests/utils/fixedpoint96.h | 2 +- tests/utils/genSmoothRandNums.h | 2 +- tests/utils/rand32.h | 2 +- tests/utils/rand64.h | 2 +- tests/utils/zfpChecksums.c | 2 +- tests/utils/zfpHash.h | 2 +- 100 files changed, 183 insertions(+), 183 deletions(-) diff --git a/include/zfp/constarray3.hpp b/include/zfp/constarray3.hpp index dbe28920d..337420762 100644 --- a/include/zfp/constarray3.hpp +++ b/include/zfp/constarray3.hpp @@ -7,13 +7,13 @@ #include "zfp/array.hpp" #include "zfp/index.hpp" #include "zfp/codec/zfpcodec.hpp" -#include "zfp/cache3.hpp" -#include "zfp/handle3.hpp" -#include "zfp/iterator3.hpp" -#include "zfp/pointer3.hpp" -#include "zfp/reference3.hpp" -#include "zfp/store3.hpp" -#include "zfp/view3.hpp" +#include "zfp/internal/array/cache3.hpp" +#include "zfp/internal/array/handle3.hpp" +#include "zfp/internal/array/iterator3.hpp" +#include "zfp/internal/array/pointer3.hpp" +#include "zfp/internal/array/reference3.hpp" +#include "zfp/internal/array/store3.hpp" +#include "zfp/internal/array/view3.hpp" namespace zfp { diff --git a/include/zfp/factory.hpp b/include/zfp/factory.hpp index c7953541b..0b3cce5bb 100644 --- a/include/zfp/factory.hpp +++ b/include/zfp/factory.hpp @@ -22,7 +22,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* std::string error; switch (dims) { case 4: -#ifdef ZFP_ARRAY4_H +#ifdef ZFP_ARRAY4_HPP switch (type) { case zfp_type_float: arr = new zfp::array4f(nx, ny, nz, nw, rate); @@ -41,7 +41,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; case 3: -#ifdef ZFP_ARRAY3_H +#ifdef ZFP_ARRAY3_HPP switch (type) { case zfp_type_float: arr = new zfp::array3f(nx, ny, nz, rate); @@ -60,7 +60,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; case 2: -#ifdef ZFP_ARRAY2_H +#ifdef ZFP_ARRAY2_HPP switch (type) { case zfp_type_float: arr = new zfp::array2f(nx, ny, rate); @@ -79,7 +79,7 @@ zfp::array* zfp::array::construct(const zfp::array::header& header, const void* break; case 1: -#ifdef ZFP_ARRAY1_H +#ifdef ZFP_ARRAY1_HPP switch (type) { case zfp_type_float: arr = new zfp::array1f(nx, rate); diff --git a/tests/array/array/testArray1d.cpp b/tests/array/array/testArray1d.cpp index e54e0929a..fc192f037 100644 --- a/tests/array/array/testArray1d.cpp +++ b/tests/array/array/testArray1d.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray1.h" -#include "array/zfparray3.h" -#include "array/zfparray4.h" -#include "array/zfpfactory.h" -#include "array/zfparray2.h" +#include "zfp/array1.hpp" +#include "zfp/array3.hpp" +#include "zfp/array4.hpp" +#include "zfp/factory.hpp" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1dIters.cpp b/tests/array/array/testArray1dIters.cpp index 49c74c8f3..18276a153 100644 --- a/tests/array/array/testArray1dIters.cpp +++ b/tests/array/array/testArray1dIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array1dTest diff --git a/tests/array/array/testArray1dPtrs.cpp b/tests/array/array/testArray1dPtrs.cpp index 8da9d65ee..be7dada8f 100644 --- a/tests/array/array/testArray1dPtrs.cpp +++ b/tests/array/array/testArray1dPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array1dTest diff --git a/tests/array/array/testArray1dRefs.cpp b/tests/array/array/testArray1dRefs.cpp index 41a13e8bf..6206dbb59 100644 --- a/tests/array/array/testArray1dRefs.cpp +++ b/tests/array/array/testArray1dRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1dViewIters.cpp b/tests/array/array/testArray1dViewIters.cpp index 63aa718b1..b00a9bced 100644 --- a/tests/array/array/testArray1dViewIters.cpp +++ b/tests/array/array/testArray1dViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1dViewPtrs.cpp b/tests/array/array/testArray1dViewPtrs.cpp index e1d48c60a..df2b35ca7 100644 --- a/tests/array/array/testArray1dViewPtrs.cpp +++ b/tests/array/array/testArray1dViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1dViews.cpp b/tests/array/array/testArray1dViews.cpp index 61e671e05..3741a1214 100644 --- a/tests/array/array/testArray1dViews.cpp +++ b/tests/array/array/testArray1dViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1f.cpp b/tests/array/array/testArray1f.cpp index f08728280..9176090c1 100644 --- a/tests/array/array/testArray1f.cpp +++ b/tests/array/array/testArray1f.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray1.h" -#include "array/zfparray3.h" -#include "array/zfparray4.h" -#include "array/zfpfactory.h" -#include "array/zfparray2.h" +#include "zfp/array1.hpp" +#include "zfp/array3.hpp" +#include "zfp/array4.hpp" +#include "zfp/factory.hpp" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1fIters.cpp b/tests/array/array/testArray1fIters.cpp index fca8e5ec0..7a8653bc4 100644 --- a/tests/array/array/testArray1fIters.cpp +++ b/tests/array/array/testArray1fIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array1fTest diff --git a/tests/array/array/testArray1fPtrs.cpp b/tests/array/array/testArray1fPtrs.cpp index a34b02a3e..cea338496 100644 --- a/tests/array/array/testArray1fPtrs.cpp +++ b/tests/array/array/testArray1fPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array1fTest diff --git a/tests/array/array/testArray1fRefs.cpp b/tests/array/array/testArray1fRefs.cpp index 517b3718f..26f2f6ed3 100644 --- a/tests/array/array/testArray1fRefs.cpp +++ b/tests/array/array/testArray1fRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1fViewIters.cpp b/tests/array/array/testArray1fViewIters.cpp index 364396fda..6a7aee3a7 100644 --- a/tests/array/array/testArray1fViewIters.cpp +++ b/tests/array/array/testArray1fViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1fViewPtrs.cpp b/tests/array/array/testArray1fViewPtrs.cpp index 8febee34b..60638ce7f 100644 --- a/tests/array/array/testArray1fViewPtrs.cpp +++ b/tests/array/array/testArray1fViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray1fViews.cpp b/tests/array/array/testArray1fViews.cpp index 142c72b72..2bad06c95 100644 --- a/tests/array/array/testArray1fViews.cpp +++ b/tests/array/array/testArray1fViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2d.cpp b/tests/array/array/testArray2d.cpp index c1d4840fd..2c1fa1c09 100644 --- a/tests/array/array/testArray2d.cpp +++ b/tests/array/array/testArray2d.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray1.h" -#include "array/zfparray2.h" -#include "array/zfparray4.h" -#include "array/zfpfactory.h" -#include "array/zfparray3.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array4.hpp" +#include "zfp/factory.hpp" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2dIters.cpp b/tests/array/array/testArray2dIters.cpp index 8f04384bc..66bb1c0ed 100644 --- a/tests/array/array/testArray2dIters.cpp +++ b/tests/array/array/testArray2dIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array2dTest diff --git a/tests/array/array/testArray2dPtrs.cpp b/tests/array/array/testArray2dPtrs.cpp index 90cf3eb94..0f0216784 100644 --- a/tests/array/array/testArray2dPtrs.cpp +++ b/tests/array/array/testArray2dPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array2dTest diff --git a/tests/array/array/testArray2dRefs.cpp b/tests/array/array/testArray2dRefs.cpp index bcfaf2d1e..ff85149c8 100644 --- a/tests/array/array/testArray2dRefs.cpp +++ b/tests/array/array/testArray2dRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2dViewIters.cpp b/tests/array/array/testArray2dViewIters.cpp index 27ba4d6b7..9e1ca9eec 100644 --- a/tests/array/array/testArray2dViewIters.cpp +++ b/tests/array/array/testArray2dViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2dViewPtrs.cpp b/tests/array/array/testArray2dViewPtrs.cpp index 3e5829f7c..008e9eb3b 100644 --- a/tests/array/array/testArray2dViewPtrs.cpp +++ b/tests/array/array/testArray2dViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2dViews.cpp b/tests/array/array/testArray2dViews.cpp index e1d40e082..ced06cf70 100644 --- a/tests/array/array/testArray2dViews.cpp +++ b/tests/array/array/testArray2dViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2f.cpp b/tests/array/array/testArray2f.cpp index d3f759526..6acb95c6c 100644 --- a/tests/array/array/testArray2f.cpp +++ b/tests/array/array/testArray2f.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray1.h" -#include "array/zfparray2.h" -#include "array/zfparray4.h" -#include "array/zfpfactory.h" -#include "array/zfparray3.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array4.hpp" +#include "zfp/factory.hpp" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2fIters.cpp b/tests/array/array/testArray2fIters.cpp index 72d82c4e7..dc5b42f3a 100644 --- a/tests/array/array/testArray2fIters.cpp +++ b/tests/array/array/testArray2fIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array2fTest diff --git a/tests/array/array/testArray2fPtrs.cpp b/tests/array/array/testArray2fPtrs.cpp index 591bcb958..bdf956cdc 100644 --- a/tests/array/array/testArray2fPtrs.cpp +++ b/tests/array/array/testArray2fPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array2fTest diff --git a/tests/array/array/testArray2fRefs.cpp b/tests/array/array/testArray2fRefs.cpp index 10fb38f4b..b3844abca 100644 --- a/tests/array/array/testArray2fRefs.cpp +++ b/tests/array/array/testArray2fRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2fViewIters.cpp b/tests/array/array/testArray2fViewIters.cpp index d082173a1..83e64552f 100644 --- a/tests/array/array/testArray2fViewIters.cpp +++ b/tests/array/array/testArray2fViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2fViewPtrs.cpp b/tests/array/array/testArray2fViewPtrs.cpp index 61dfee132..b0e9f2d34 100644 --- a/tests/array/array/testArray2fViewPtrs.cpp +++ b/tests/array/array/testArray2fViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray2fViews.cpp b/tests/array/array/testArray2fViews.cpp index e43eb69c9..891eb1f8d 100644 --- a/tests/array/array/testArray2fViews.cpp +++ b/tests/array/array/testArray2fViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray2.h" +#include "zfp/array2.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3d.cpp b/tests/array/array/testArray3d.cpp index db9ddb3aa..0287f4ad5 100644 --- a/tests/array/array/testArray3d.cpp +++ b/tests/array/array/testArray3d.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray1.h" -#include "array/zfparray2.h" -#include "array/zfparray3.h" -#include "array/zfpfactory.h" -#include "array/zfparray4.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/factory.hpp" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3dIters.cpp b/tests/array/array/testArray3dIters.cpp index b363483de..3c096c082 100644 --- a/tests/array/array/testArray3dIters.cpp +++ b/tests/array/array/testArray3dIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array3dTest diff --git a/tests/array/array/testArray3dPtrs.cpp b/tests/array/array/testArray3dPtrs.cpp index 0d7050024..ebf9ac6ac 100644 --- a/tests/array/array/testArray3dPtrs.cpp +++ b/tests/array/array/testArray3dPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array3dTest diff --git a/tests/array/array/testArray3dRefs.cpp b/tests/array/array/testArray3dRefs.cpp index d4a43582b..9e47931b4 100644 --- a/tests/array/array/testArray3dRefs.cpp +++ b/tests/array/array/testArray3dRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3dViewIters.cpp b/tests/array/array/testArray3dViewIters.cpp index 40f5d6c5e..fcdd65a97 100644 --- a/tests/array/array/testArray3dViewIters.cpp +++ b/tests/array/array/testArray3dViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3dViewPtrs.cpp b/tests/array/array/testArray3dViewPtrs.cpp index b4e37c4f0..5eafb7690 100644 --- a/tests/array/array/testArray3dViewPtrs.cpp +++ b/tests/array/array/testArray3dViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3dViews.cpp b/tests/array/array/testArray3dViews.cpp index cda31375a..117a49c42 100644 --- a/tests/array/array/testArray3dViews.cpp +++ b/tests/array/array/testArray3dViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3f.cpp b/tests/array/array/testArray3f.cpp index ae8112ad8..22bb3d45c 100644 --- a/tests/array/array/testArray3f.cpp +++ b/tests/array/array/testArray3f.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray1.h" -#include "array/zfparray2.h" -#include "array/zfparray3.h" -#include "array/zfpfactory.h" -#include "array/zfparray4.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/factory.hpp" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3fIters.cpp b/tests/array/array/testArray3fIters.cpp index 676e1f7fe..4eaf6376e 100644 --- a/tests/array/array/testArray3fIters.cpp +++ b/tests/array/array/testArray3fIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array3fTest diff --git a/tests/array/array/testArray3fPtrs.cpp b/tests/array/array/testArray3fPtrs.cpp index d6e3698b5..d1a7801c4 100644 --- a/tests/array/array/testArray3fPtrs.cpp +++ b/tests/array/array/testArray3fPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array3fTest diff --git a/tests/array/array/testArray3fRefs.cpp b/tests/array/array/testArray3fRefs.cpp index f6afbfed0..1f27c1780 100644 --- a/tests/array/array/testArray3fRefs.cpp +++ b/tests/array/array/testArray3fRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3fViewIters.cpp b/tests/array/array/testArray3fViewIters.cpp index 03a1d881a..74a56fb3d 100644 --- a/tests/array/array/testArray3fViewIters.cpp +++ b/tests/array/array/testArray3fViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3fViewPtrs.cpp b/tests/array/array/testArray3fViewPtrs.cpp index e5641050d..fa14e6812 100644 --- a/tests/array/array/testArray3fViewPtrs.cpp +++ b/tests/array/array/testArray3fViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray3fViews.cpp b/tests/array/array/testArray3fViews.cpp index 533c24aba..7ddd08745 100644 --- a/tests/array/array/testArray3fViews.cpp +++ b/tests/array/array/testArray3fViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4d.cpp b/tests/array/array/testArray4d.cpp index 38a22df56..5fb688f1d 100644 --- a/tests/array/array/testArray4d.cpp +++ b/tests/array/array/testArray4d.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray2.h" -#include "array/zfparray3.h" -#include "array/zfparray4.h" -#include "array/zfpfactory.h" -#include "array/zfparray1.h" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/array4.hpp" +#include "zfp/factory.hpp" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4dIters.cpp b/tests/array/array/testArray4dIters.cpp index 12ff9a458..9be8d7c8d 100644 --- a/tests/array/array/testArray4dIters.cpp +++ b/tests/array/array/testArray4dIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array4dTest diff --git a/tests/array/array/testArray4dPtrs.cpp b/tests/array/array/testArray4dPtrs.cpp index 1c43f54ab..5eae996a9 100644 --- a/tests/array/array/testArray4dPtrs.cpp +++ b/tests/array/array/testArray4dPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array4dTest diff --git a/tests/array/array/testArray4dRefs.cpp b/tests/array/array/testArray4dRefs.cpp index ceeca2ef0..4560c83b4 100644 --- a/tests/array/array/testArray4dRefs.cpp +++ b/tests/array/array/testArray4dRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4dViewIters.cpp b/tests/array/array/testArray4dViewIters.cpp index bc85b3757..d0be737cd 100644 --- a/tests/array/array/testArray4dViewIters.cpp +++ b/tests/array/array/testArray4dViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4dViewPtrs.cpp b/tests/array/array/testArray4dViewPtrs.cpp index 6e537b588..0eee3c464 100644 --- a/tests/array/array/testArray4dViewPtrs.cpp +++ b/tests/array/array/testArray4dViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4dViews.cpp b/tests/array/array/testArray4dViews.cpp index 427499c18..d53c3cd1e 100644 --- a/tests/array/array/testArray4dViews.cpp +++ b/tests/array/array/testArray4dViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4f.cpp b/tests/array/array/testArray4f.cpp index d908487dd..dbd8c0707 100644 --- a/tests/array/array/testArray4f.cpp +++ b/tests/array/array/testArray4f.cpp @@ -1,8 +1,8 @@ -#include "array/zfparray2.h" -#include "array/zfparray3.h" -#include "array/zfparray4.h" -#include "array/zfpfactory.h" -#include "array/zfparray1.h" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/array4.hpp" +#include "zfp/factory.hpp" +#include "zfp/array1.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4fIters.cpp b/tests/array/array/testArray4fIters.cpp index 2a03e9785..ddcdb42bd 100644 --- a/tests/array/array/testArray4fIters.cpp +++ b/tests/array/array/testArray4fIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array4fTest diff --git a/tests/array/array/testArray4fPtrs.cpp b/tests/array/array/testArray4fPtrs.cpp index 7b6fb8352..7a726b841 100644 --- a/tests/array/array/testArray4fPtrs.cpp +++ b/tests/array/array/testArray4fPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; #define ARRAY_DIMS_SCALAR_TEST Array4fTest diff --git a/tests/array/array/testArray4fRefs.cpp b/tests/array/array/testArray4fRefs.cpp index aec1c8930..c20c305bc 100644 --- a/tests/array/array/testArray4fRefs.cpp +++ b/tests/array/array/testArray4fRefs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4fViewIters.cpp b/tests/array/array/testArray4fViewIters.cpp index 3cca047a1..28f6fb7ea 100644 --- a/tests/array/array/testArray4fViewIters.cpp +++ b/tests/array/array/testArray4fViewIters.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4fViewPtrs.cpp b/tests/array/array/testArray4fViewPtrs.cpp index b454ca770..b4a475a94 100644 --- a/tests/array/array/testArray4fViewPtrs.cpp +++ b/tests/array/array/testArray4fViewPtrs.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArray4fViews.cpp b/tests/array/array/testArray4fViews.cpp index 6c648f303..c0e44137a 100644 --- a/tests/array/array/testArray4fViews.cpp +++ b/tests/array/array/testArray4fViews.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray4.h" +#include "zfp/array4.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/array/testArrayBase.cpp b/tests/array/array/testArrayBase.cpp index 30edc07be..d0ca5afee 100644 --- a/tests/array/array/testArrayBase.cpp +++ b/tests/array/array/testArrayBase.cpp @@ -221,7 +221,7 @@ TEST_F(TEST_FIXTURE, given_zfpHeaderForCertainDimensionalityButHeaderMissing_whe } catch (zfp::exception const & e) { std::stringstream ss; - ss << "zfparray" << missingDim << " not supported; include zfparray" << missingDim << ".h before zfpfactory.h"; + ss << "array" << missingDim << " not supported; include zfp/array" << missingDim << ".hpp before zfp/factory.hpp"; EXPECT_EQ(e.what(), ss.str()); } catch (std::exception const & e) { diff --git a/tests/array/array/testConstruct.cpp b/tests/array/array/testConstruct.cpp index 336a24336..acf153f78 100644 --- a/tests/array/array/testConstruct.cpp +++ b/tests/array/array/testConstruct.cpp @@ -1,6 +1,6 @@ -#include "array/zfparray2.h" -#include "array/zfparray3.h" -#include "array/zfpfactory.h" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/factory.hpp" using namespace zfp; #include "gtest/gtest.h" @@ -70,7 +70,7 @@ TEST_F(TEST_FIXTURE, given_onlyInclude2D3D_and_zfpHeaderFor1D_when_construct_exp zfp::array* arr = zfp::array::construct(h); FailWhenNoExceptionThrown(); } catch (zfp::exception const & e) { - EXPECT_EQ(e.what(), std::string("zfparray1 not supported; include zfparray1.h before zfpfactory.h")); + EXPECT_EQ(e.what(), std::string("array1 not supported; include zfp/array1.hpp before zfp/factory.hpp")); } catch (std::exception const & e) { FailAndPrintException(e); } diff --git a/tests/array/constArray/testConstArray1d.cpp b/tests/array/constArray/testConstArray1d.cpp index 2b3bd5f42..d1e8edd22 100644 --- a/tests/array/constArray/testConstArray1d.cpp +++ b/tests/array/constArray/testConstArray1d.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray1f.cpp b/tests/array/constArray/testConstArray1f.cpp index ec2676170..321d5d20b 100644 --- a/tests/array/constArray/testConstArray1f.cpp +++ b/tests/array/constArray/testConstArray1f.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray2d.cpp b/tests/array/constArray/testConstArray2d.cpp index 02bee1e86..e92360f18 100644 --- a/tests/array/constArray/testConstArray2d.cpp +++ b/tests/array/constArray/testConstArray2d.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray2f.cpp b/tests/array/constArray/testConstArray2f.cpp index f68ee0f4c..bde637098 100644 --- a/tests/array/constArray/testConstArray2f.cpp +++ b/tests/array/constArray/testConstArray2f.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray3d.cpp b/tests/array/constArray/testConstArray3d.cpp index 2f4d062c9..59c091eef 100644 --- a/tests/array/constArray/testConstArray3d.cpp +++ b/tests/array/constArray/testConstArray3d.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray3f.cpp b/tests/array/constArray/testConstArray3f.cpp index e4f795841..44358955c 100644 --- a/tests/array/constArray/testConstArray3f.cpp +++ b/tests/array/constArray/testConstArray3f.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray4d.cpp b/tests/array/constArray/testConstArray4d.cpp index c238b7afc..598417e96 100644 --- a/tests/array/constArray/testConstArray4d.cpp +++ b/tests/array/constArray/testConstArray4d.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/constArray/testConstArray4f.cpp b/tests/array/constArray/testConstArray4f.cpp index 00561e8bd..cf5f08d59 100644 --- a/tests/array/constArray/testConstArray4f.cpp +++ b/tests/array/constArray/testConstArray4f.cpp @@ -1,8 +1,8 @@ -#include "array/zfpcarray1.h" -#include "array/zfpcarray2.h" -#include "array/zfpcarray3.h" -#include "array/zfpcarray4.h" -#include "array/zfpfactory.h" +#include "zfp/constarray1.hpp" +#include "zfp/constarray2.hpp" +#include "zfp/constarray3.hpp" +#include "zfp/constarray4.hpp" +#include "zfp/factory.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode1d.cpp b/tests/array/decode/testTemplatedDecode1d.cpp index 05d034acb..ca70fd89c 100644 --- a/tests/array/decode/testTemplatedDecode1d.cpp +++ b/tests/array/decode/testTemplatedDecode1d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode1f.cpp b/tests/array/decode/testTemplatedDecode1f.cpp index 03f2c6c34..914b3c95c 100644 --- a/tests/array/decode/testTemplatedDecode1f.cpp +++ b/tests/array/decode/testTemplatedDecode1f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode2d.cpp b/tests/array/decode/testTemplatedDecode2d.cpp index 87d816504..5915f10fc 100644 --- a/tests/array/decode/testTemplatedDecode2d.cpp +++ b/tests/array/decode/testTemplatedDecode2d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode2f.cpp b/tests/array/decode/testTemplatedDecode2f.cpp index 61fe0b4af..526b2bd04 100644 --- a/tests/array/decode/testTemplatedDecode2f.cpp +++ b/tests/array/decode/testTemplatedDecode2f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode3d.cpp b/tests/array/decode/testTemplatedDecode3d.cpp index 881231907..4c567e90e 100644 --- a/tests/array/decode/testTemplatedDecode3d.cpp +++ b/tests/array/decode/testTemplatedDecode3d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode3f.cpp b/tests/array/decode/testTemplatedDecode3f.cpp index daa336f46..4fc28e944 100644 --- a/tests/array/decode/testTemplatedDecode3f.cpp +++ b/tests/array/decode/testTemplatedDecode3f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode4d.cpp b/tests/array/decode/testTemplatedDecode4d.cpp index 1fa9d34f7..c159c49c8 100644 --- a/tests/array/decode/testTemplatedDecode4d.cpp +++ b/tests/array/decode/testTemplatedDecode4d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/decode/testTemplatedDecode4f.cpp b/tests/array/decode/testTemplatedDecode4f.cpp index 135ef4908..b94b32a01 100644 --- a/tests/array/decode/testTemplatedDecode4f.cpp +++ b/tests/array/decode/testTemplatedDecode4f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode1d.cpp b/tests/array/encode/testTemplatedEncode1d.cpp index f1ef6b261..e8e1a040d 100644 --- a/tests/array/encode/testTemplatedEncode1d.cpp +++ b/tests/array/encode/testTemplatedEncode1d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode1f.cpp b/tests/array/encode/testTemplatedEncode1f.cpp index 0127f6a66..f30dcdba4 100644 --- a/tests/array/encode/testTemplatedEncode1f.cpp +++ b/tests/array/encode/testTemplatedEncode1f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode2d.cpp b/tests/array/encode/testTemplatedEncode2d.cpp index 3412abb5c..c853ea3d5 100644 --- a/tests/array/encode/testTemplatedEncode2d.cpp +++ b/tests/array/encode/testTemplatedEncode2d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode2f.cpp b/tests/array/encode/testTemplatedEncode2f.cpp index 35a11dab5..6e4ea5ba2 100644 --- a/tests/array/encode/testTemplatedEncode2f.cpp +++ b/tests/array/encode/testTemplatedEncode2f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode3d.cpp b/tests/array/encode/testTemplatedEncode3d.cpp index bced5bc6f..6260e659a 100644 --- a/tests/array/encode/testTemplatedEncode3d.cpp +++ b/tests/array/encode/testTemplatedEncode3d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode3f.cpp b/tests/array/encode/testTemplatedEncode3f.cpp index a26fc3f9c..af2079fa3 100644 --- a/tests/array/encode/testTemplatedEncode3f.cpp +++ b/tests/array/encode/testTemplatedEncode3f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode4d.cpp b/tests/array/encode/testTemplatedEncode4d.cpp index b1283e3b0..c71a27e43 100644 --- a/tests/array/encode/testTemplatedEncode4d.cpp +++ b/tests/array/encode/testTemplatedEncode4d.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/encode/testTemplatedEncode4f.cpp b/tests/array/encode/testTemplatedEncode4f.cpp index c63dcefd5..91202a741 100644 --- a/tests/array/encode/testTemplatedEncode4f.cpp +++ b/tests/array/encode/testTemplatedEncode4f.cpp @@ -1,4 +1,4 @@ -#include "array/zfpcpp.h" +#include "zfp.hpp" using namespace zfp; extern "C" { diff --git a/tests/array/utils/predicates.h b/tests/array/utils/predicates.h index 9ba726157..347143d96 100644 --- a/tests/array/utils/predicates.h +++ b/tests/array/utils/predicates.h @@ -1,4 +1,4 @@ -#include "array/zfparray1.h" +#include "zfp/array1.hpp" #include "gtest/gtest.h" diff --git a/tests/array/zfp/testAlignedMemory.cpp b/tests/array/zfp/testAlignedMemory.cpp index 54871a5d9..0c94a5d21 100644 --- a/tests/array/zfp/testAlignedMemory.cpp +++ b/tests/array/zfp/testAlignedMemory.cpp @@ -1,4 +1,4 @@ -#include "array/zfparray3.h" +#include "zfp/array3.hpp" using namespace zfp; #include "gtest/gtest.h" diff --git a/tests/cfp/testCfpArray_source.c b/tests/cfp/testCfpArray_source.c index 718812530..6731ebbda 100644 --- a/tests/cfp/testCfpArray_source.c +++ b/tests/cfp/testCfpArray_source.c @@ -8,7 +8,7 @@ #include #include -#include "cfparray.h" +#include "zfp/cfp.h" #include "zfp.h" #include "utils/genSmoothRandNums.h" diff --git a/tests/cfp/testCfpNamespace.c b/tests/cfp/testCfpNamespace.c index 478c27bca..3e341cb10 100644 --- a/tests/cfp/testCfpNamespace.c +++ b/tests/cfp/testCfpNamespace.c @@ -3,7 +3,7 @@ #include #include -#include "cfparray.h" +#include "zfp/cfp.h" /* only run this test when compiling with CFP_NAMESPACE=cfp2 */ diff --git a/tests/src/execPolicy/testOmpInternal.c b/tests/src/execPolicy/testOmpInternal.c index b8eac13ef..dfa89eb57 100644 --- a/tests/src/execPolicy/testOmpInternal.c +++ b/tests/src/execPolicy/testOmpInternal.c @@ -1,5 +1,5 @@ #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" #include "src/share/omp.c" #include diff --git a/tests/src/inline/testBitstream.c b/tests/src/inline/testBitstream.c index a8ac4cd76..c025d95f9 100644 --- a/tests/src/inline/testBitstream.c +++ b/tests/src/inline/testBitstream.c @@ -3,8 +3,8 @@ #include #include -#include "include/bitstream.h" -#include "src/inline/bitstream.c" +#include "include/zfp/bitstream.h" +#include "include/zfp/bitstream.inl" #define STREAM_WORD_CAPACITY 3 diff --git a/tests/src/inline/testBitstreamSmallWsize.c b/tests/src/inline/testBitstreamSmallWsize.c index 1c4f96295..23350e553 100644 --- a/tests/src/inline/testBitstreamSmallWsize.c +++ b/tests/src/inline/testBitstreamSmallWsize.c @@ -5,8 +5,8 @@ #define BIT_STREAM_WORD_TYPE uint16 -#include "include/bitstream.h" -#include "src/inline/bitstream.c" +#include "include/zfp/bitstream.h" +#include "include/zfp/bitstream.inl" #define STREAM_WORD_CAPACITY 4 diff --git a/tests/src/inline/testBitstreamStrided.c b/tests/src/inline/testBitstreamStrided.c index a7fa2c7a3..d628d9eb0 100644 --- a/tests/src/inline/testBitstreamStrided.c +++ b/tests/src/inline/testBitstreamStrided.c @@ -5,8 +5,8 @@ #define BIT_STREAM_STRIDED -#include "include/bitstream.h" -#include "src/inline/bitstream.c" +#include "include/zfp/bitstream.h" +#include "include/zfp/bitstream.inl" // 4 words per block #define BLOCK_SIZE 4 diff --git a/tests/testviews.cpp b/tests/testviews.cpp index 3827c5265..a90210d51 100644 --- a/tests/testviews.cpp +++ b/tests/testviews.cpp @@ -2,8 +2,8 @@ #include #include #include -#include "zfparray2.h" -#include "zfparray3.h" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" #define EPSILON 1e-3 diff --git a/tests/testzfp.cpp b/tests/testzfp.cpp index 9c0bdb7f4..21c26e174 100644 --- a/tests/testzfp.cpp +++ b/tests/testzfp.cpp @@ -11,10 +11,10 @@ #include #include #include "zfp.h" -#include "zfparray1.h" -#include "zfparray2.h" -#include "zfparray3.h" -#include "zfparray4.h" +#include "zfp/array1.hpp" +#include "zfp/array2.hpp" +#include "zfp/array3.hpp" +#include "zfp/array4.hpp" enum ArraySize { Small = 0, // 2^12 = 4096 scalars (2^12 = (2^6)^2 = (2^4)^3 = (2^3)^4) diff --git a/tests/utils/fixedpoint96.h b/tests/utils/fixedpoint96.h index 7a701b747..b48ebdab5 100644 --- a/tests/utils/fixedpoint96.h +++ b/tests/utils/fixedpoint96.h @@ -1,7 +1,7 @@ #ifndef FIXEDPT_H #define FIXEDPT_H -#include "include/zfp/types.h" +#include "include/zfp/internal/zfp/types.h" typedef struct { // the number represented = i + (2^-32)*f diff --git a/tests/utils/genSmoothRandNums.h b/tests/utils/genSmoothRandNums.h index b8969d8f1..9d43c72fa 100644 --- a/tests/utils/genSmoothRandNums.h +++ b/tests/utils/genSmoothRandNums.h @@ -1,7 +1,7 @@ #ifndef GEN_SMOOTH_RAND_INTS_H #define GEN_SMOOTH_RAND_INTS_H -#include "include/zfp/types.h" +#include "zfp/internal/zfp/types.h" // used to compute (square) array sizes size_t diff --git a/tests/utils/rand32.h b/tests/utils/rand32.h index de82cce6b..a47328db3 100644 --- a/tests/utils/rand32.h +++ b/tests/utils/rand32.h @@ -1,7 +1,7 @@ #ifndef RAND_32_H #define RAND_32_H -#include "include/zfp/types.h" +#include "include/zfp/internal/zfp/types.h" // reset seed void diff --git a/tests/utils/rand64.h b/tests/utils/rand64.h index 78c526184..4c4ec161a 100644 --- a/tests/utils/rand64.h +++ b/tests/utils/rand64.h @@ -1,7 +1,7 @@ #ifndef RAND_64_H #define RAND_64_H -#include "include/zfp/types.h" +#include "include/zfp/internal/zfp/types.h" // reset seed void diff --git a/tests/utils/zfpChecksums.c b/tests/utils/zfpChecksums.c index cf9b7e62e..f9b1f7376 100644 --- a/tests/utils/zfpChecksums.c +++ b/tests/utils/zfpChecksums.c @@ -1,4 +1,4 @@ -#include "zfp/types.h" +#include "zfp/internal/zfp/types.h" #include "zfpChecksums.h" #define NUM_INT_CHECKSUMS 19 diff --git a/tests/utils/zfpHash.h b/tests/utils/zfpHash.h index 446065725..5718050f3 100644 --- a/tests/utils/zfpHash.h +++ b/tests/utils/zfpHash.h @@ -2,7 +2,7 @@ #define ZFP_HASH_H #include -#include "include/zfp/types.h" +#include "include/zfp/internal/zfp/types.h" uint64 hashBitstream(uint64* ptrStart, size_t bufsizeBytes); From 5fd6de39fe7c567b2b969e37981f90396e4e915f Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 7 Jul 2022 10:39:11 -0700 Subject: [PATCH 075/277] update zfp util --- utils/zfp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/zfp.c b/utils/zfp.c index 79608db77..3520f7437 100644 --- a/utils/zfp.c +++ b/utils/zfp.c @@ -5,7 +5,7 @@ #include #include #include "zfp.h" -#include "zfp/macros.h" +#include "zfp/internal/zfp/macros.h" /* File I/O is done using the following combinations of i, o, s, and z: From 3674669dffcfa2db82e424cb8b71accd360c579f Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 7 Jul 2022 11:02:15 -0700 Subject: [PATCH 076/277] Update zforp install location --- fortran/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fortran/CMakeLists.txt b/fortran/CMakeLists.txt index a3e6b9c2c..51799401b 100644 --- a/fortran/CMakeLists.txt +++ b/fortran/CMakeLists.txt @@ -22,7 +22,7 @@ set_property(TARGET zFORp PROPERTY OUTPUT_NAME ${ZFP_LIBRARY_PREFIX}zFORp) # install location for module file install(FILES ${CMAKE_Fortran_MODULE_DIRECTORY}/zforp.mod - DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) + DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/zfp) # install location for library install(TARGETS zFORp EXPORT cFORp-targets From 9e78178a44c2decb8aefc3ff520e5ca2a0bc9f75 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 7 Jul 2022 11:07:55 -0700 Subject: [PATCH 077/277] Fix include in zfpy --- python/zfpy.pxd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/zfpy.pxd b/python/zfpy.pxd index 60922c6b8..4d89776b8 100644 --- a/python/zfpy.pxd +++ b/python/zfpy.pxd @@ -1,7 +1,7 @@ import cython cimport libc.stdint as stdint -cdef extern from "bitstream.h": +cdef extern from "zfp/bitstream.h": cdef struct bitstream: pass bitstream* stream_open(void* data, size_t) From c13a6c7b91199d2446b1c8a132d17893ce55a800 Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Thu, 7 Jul 2022 11:29:39 -0700 Subject: [PATCH 078/277] Update bitstream.c include in cuda [skip ci] --- src/cuda_zfp/cuZFP.cu | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cuda_zfp/cuZFP.cu b/src/cuda_zfp/cuZFP.cu index 619caca22..fe1ba7248 100644 --- a/src/cuda_zfp/cuZFP.cu +++ b/src/cuda_zfp/cuZFP.cu @@ -23,7 +23,7 @@ #define inline_ inline #endif -#include "../inline/bitstream.c" +#include "zfp/bitstream.inl" namespace internal { From 05b3f4c56bf106e627280f7d75c9f144030712ac Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 13 Jul 2022 11:50:30 -0700 Subject: [PATCH 079/277] Update fortran module name and move internal codec headers --- cfp/cfpheader.cpp | 2 +- fortran/CMakeLists.txt | 4 ++-- fortran/zfp.f90 | 4 ++-- include/zfp/codec/zfpcodec.hpp | 2 +- include/zfp/internal/{array => codec}/zfpheader.hpp | 0 5 files changed, 6 insertions(+), 6 deletions(-) rename include/zfp/internal/{array => codec}/zfpheader.hpp (100%) diff --git a/cfp/cfpheader.cpp b/cfp/cfpheader.cpp index efadd8677..b4b66e09e 100644 --- a/cfp/cfpheader.cpp +++ b/cfp/cfpheader.cpp @@ -2,7 +2,7 @@ #include "zfp/array2.hpp" #include "zfp/array3.hpp" #include "zfp/array4.hpp" -#include "zfp/internal/array/zfpheader.hpp" +#include "zfp/internal/codec/zfpheader.hpp" #include "zfp/internal/cfp/header.h" #include "zfp/internal/cfp/array1f.h" #include "zfp/internal/cfp/array1d.h" diff --git a/fortran/CMakeLists.txt b/fortran/CMakeLists.txt index 51799401b..9c3763489 100644 --- a/fortran/CMakeLists.txt +++ b/fortran/CMakeLists.txt @@ -21,8 +21,8 @@ set_property(TARGET zFORp PROPERTY SOVERSION ${ZFP_VERSION_MAJOR}) set_property(TARGET zFORp PROPERTY OUTPUT_NAME ${ZFP_LIBRARY_PREFIX}zFORp) # install location for module file -install(FILES ${CMAKE_Fortran_MODULE_DIRECTORY}/zforp.mod - DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/zfp) +install(FILES ${CMAKE_Fortran_MODULE_DIRECTORY}/zfp.mod + DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) # install location for library install(TARGETS zFORp EXPORT cFORp-targets diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index 8042f2159..8b884a5ee 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -1,4 +1,4 @@ -module zFORp +module zfp use, intrinsic :: iso_c_binding, only: c_int, c_int64_t, c_size_t, c_ptrdiff_t, c_double, c_ptr, c_null_ptr, c_loc implicit none @@ -1113,4 +1113,4 @@ function zFORp_read_header(stream, field, mask) result(num_bits_read) bind(c, na num_bits_read = zfp_read_header(stream%object, field%object, int(mask, c_int)) end function zFORp_read_header -end module zFORp +end module zfp diff --git a/include/zfp/codec/zfpcodec.hpp b/include/zfp/codec/zfpcodec.hpp index 96399b710..fd9f4be19 100644 --- a/include/zfp/codec/zfpcodec.hpp +++ b/include/zfp/codec/zfpcodec.hpp @@ -115,7 +115,7 @@ class zfp_base { static const zfp_type type = zfp::trait::type; // scalar type // zfp::codec::zfp_base::header class for array (de)serialization - #include "zfp/internal/array/zfpheader.hpp" + #include "zfp/internal/codec/zfpheader.hpp" protected: // deep copy diff --git a/include/zfp/internal/array/zfpheader.hpp b/include/zfp/internal/codec/zfpheader.hpp similarity index 100% rename from include/zfp/internal/array/zfpheader.hpp rename to include/zfp/internal/codec/zfpheader.hpp From 20e62c58282d2bd72a202d1f9c15dcb49e5bbef6 Mon Sep 17 00:00:00 2001 From: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com> Date: Thu, 14 Jul 2022 13:12:01 +0200 Subject: [PATCH 080/277] Fix typos found by codespell --- array/zfp/handle1.h | 2 +- array/zfp/handle2.h | 2 +- array/zfp/handle3.h | 2 +- array/zfp/handle4.h | 2 +- docs/source/arrays.rst | 2 +- python/CMakeLists.txt | 2 +- src/cuda_zfp/cuZFP.cu | 2 +- src/cuda_zfp/decode.cuh | 2 +- src/cuda_zfp/decode1.cuh | 6 +++--- src/cuda_zfp/decode2.cuh | 6 +++--- src/cuda_zfp/decode3.cuh | 6 +++--- src/cuda_zfp/encode1.cuh | 6 +++--- src/cuda_zfp/encode2.cuh | 6 +++--- src/cuda_zfp/encode3.cuh | 6 +++--- src/template/encode.c | 2 +- tests/src/endtoend/serialExecBase.c | 2 +- 16 files changed, 28 insertions(+), 28 deletions(-) diff --git a/array/zfp/handle1.h b/array/zfp/handle1.h index 4174fad69..5b4d4b5be 100644 --- a/array/zfp/handle1.h +++ b/array/zfp/handle1.h @@ -24,7 +24,7 @@ class const_handle { // protected constructor explicit const_handle(const container_type* container, size_t x) : container(const_cast(container)), x(x) {} - // derefence handle + // dereference handle value_type get() const { return container->get(x); } container_type* container; // container diff --git a/array/zfp/handle2.h b/array/zfp/handle2.h index 81feead11..4958c9bbc 100644 --- a/array/zfp/handle2.h +++ b/array/zfp/handle2.h @@ -24,7 +24,7 @@ class const_handle { // protected constructor explicit const_handle(const container_type* container, size_t x, size_t y) : container(const_cast(container)), x(x), y(y) {} - // derefence handle + // dereference handle value_type get() const { return container->get(x, y); } container_type* container; // container diff --git a/array/zfp/handle3.h b/array/zfp/handle3.h index 85e09d5cf..3fb1920c3 100644 --- a/array/zfp/handle3.h +++ b/array/zfp/handle3.h @@ -24,7 +24,7 @@ class const_handle { // protected constructor explicit const_handle(const container_type* container, size_t x, size_t y, size_t z) : container(const_cast(container)), x(x), y(y), z(z) {} - // derefence handle + // dereference handle value_type get() const { return container->get(x, y, z); } container_type* container; // container diff --git a/array/zfp/handle4.h b/array/zfp/handle4.h index 3616344f6..56f1a9c0d 100644 --- a/array/zfp/handle4.h +++ b/array/zfp/handle4.h @@ -24,7 +24,7 @@ class const_handle { // protected constructor explicit const_handle(const container_type* container, size_t x, size_t y, size_t z, size_t w) : container(const_cast(container)), x(x), y(y), z(z), w(w) {} - // derefence handle + // dereference handle value_type get() const { return container->get(x, y, z, w); } container_type* container; // container diff --git a/docs/source/arrays.rst b/docs/source/arrays.rst index f0574534f..c1c52d45e 100644 --- a/docs/source/arrays.rst +++ b/docs/source/arrays.rst @@ -716,7 +716,7 @@ Additional methods are documented below. Whereas the :ref:`read-write fixed-rate arrays ` (:cpp:class:`zfp::array`) require that block storage is word aligned, the read-only arrays (:cpp:class:`zfp::const_array`) are not subject to such - restrictions and thefore support finer rate granularity. For a + restrictions and therefore support finer rate granularity. For a *d*-dimensional :cpp:class:`const_array`, the rate granularity is 4\ :sup:`-d` bits/value, e.g., a quarter bit/value for 1D arrays. diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index c6f6e27c0..9c06d453c 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -17,7 +17,7 @@ add_library(zfpy MODULE ${zfpy}) target_link_libraries(zfpy zfp) python_extension_module(zfpy) -# Build to the currrent binary dir to avoid conflicts with other libraries named zfp +# Build to the current binary dir to avoid conflicts with other libraries named zfp set(PYLIB_BUILD_DIR "${CMAKE_BINARY_DIR}/bin" CACHE PATH "Directory where zfp python library will be built") set_target_properties(zfpy PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${PYLIB_BUILD_DIR}) diff --git a/src/cuda_zfp/cuZFP.cu b/src/cuda_zfp/cuZFP.cu index 619caca22..49703c5b7 100644 --- a/src/cuda_zfp/cuZFP.cu +++ b/src/cuda_zfp/cuZFP.cu @@ -483,7 +483,7 @@ cuda_decompress(zfp_stream *stream, zfp_field *field) internal::cleanup_device_ptr(stream->stream->begin, d_stream, 0, 0, field->type); internal::cleanup_device_ptr(field->data, d_data, bytes, offset, field->type); - // this is how zfp determins if this was a success + // this is how zfp determines if this was a success size_t words_read = decoded_bytes / sizeof(Word); stream->stream->bits = wsize; // set stream pointer to end of stream diff --git a/src/cuda_zfp/decode.cuh b/src/cuda_zfp/decode.cuh index 07e9866ea..636de7d4e 100644 --- a/src/cuda_zfp/decode.cuh +++ b/src/cuda_zfp/decode.cuh @@ -112,7 +112,7 @@ public: next_read = n_bits - first_read; } - // this is basically a no-op when first read constained + // this is basically a no-op when first read contained // all the bits. TODO: if we have aligned reads, this could // be a conditional without divergence mask = ((Word)1<<((next_read)))-1; diff --git a/src/cuda_zfp/decode1.cuh b/src/cuda_zfp/decode1.cuh index b1f474d19..6d357f631 100644 --- a/src/cuda_zfp/decode1.cuh +++ b/src/cuda_zfp/decode1.cuh @@ -127,9 +127,9 @@ size_t decode1launch(uint dim, cudaEventSynchronize(stop); cudaStreamSynchronize(0); - float miliseconds = 0; - cudaEventElapsedTime(&miliseconds, start, stop); - float seconds = miliseconds / 1000.f; + float milliseconds = 0; + cudaEventElapsedTime(&milliseconds, start, stop); + float seconds = milliseconds / 1000.f; float rate = (float(dim) * sizeof(Scalar) ) / seconds; rate /= 1024.f; rate /= 1024.f; diff --git a/src/cuda_zfp/decode2.cuh b/src/cuda_zfp/decode2.cuh index 3378f5a56..fa60a82f7 100644 --- a/src/cuda_zfp/decode2.cuh +++ b/src/cuda_zfp/decode2.cuh @@ -150,9 +150,9 @@ size_t decode2launch(uint2 dims, cudaEventSynchronize(stop); cudaStreamSynchronize(0); - float miliseconds = 0; - cudaEventElapsedTime(&miliseconds, start, stop); - float seconds = miliseconds / 1000.f; + float milliseconds = 0; + cudaEventElapsedTime(&milliseconds, start, stop); + float seconds = milliseconds / 1000.f; float rate = (float(dims.x * dims.y) * sizeof(Scalar) ) / seconds; rate /= 1024.f; rate /= 1024.f; diff --git a/src/cuda_zfp/decode3.cuh b/src/cuda_zfp/decode3.cuh index 7092f9a3f..9f2a98a89 100644 --- a/src/cuda_zfp/decode3.cuh +++ b/src/cuda_zfp/decode3.cuh @@ -163,9 +163,9 @@ size_t decode3launch(uint3 dims, cudaEventSynchronize(stop); cudaStreamSynchronize(0); - float miliseconds = 0; - cudaEventElapsedTime(&miliseconds, start, stop); - float seconds = miliseconds / 1000.f; + float milliseconds = 0; + cudaEventElapsedTime(&milliseconds, start, stop); + float seconds = milliseconds / 1000.f; float rate = (float(dims.x * dims.y * dims.z) * sizeof(Scalar) ) / seconds; rate /= 1024.f; rate /= 1024.f; diff --git a/src/cuda_zfp/encode1.cuh b/src/cuda_zfp/encode1.cuh index 8ef37b14f..98ce5a753 100644 --- a/src/cuda_zfp/encode1.cuh +++ b/src/cuda_zfp/encode1.cuh @@ -145,9 +145,9 @@ size_t encode1launch(uint dim, cudaEventSynchronize(stop); cudaStreamSynchronize(0); - float miliseconds = 0.f; - cudaEventElapsedTime(&miliseconds, start, stop); - float seconds = miliseconds / 1000.f; + float milliseconds = 0.f; + cudaEventElapsedTime(&milliseconds, start, stop); + float seconds = milliseconds / 1000.f; float gb = (float(dim) * float(sizeof(Scalar))) / (1024.f * 1024.f * 1024.f); float rate = gb / seconds; printf("Encode elapsed time: %.5f (s)\n", seconds); diff --git a/src/cuda_zfp/encode2.cuh b/src/cuda_zfp/encode2.cuh index 105750e43..0d577d51e 100644 --- a/src/cuda_zfp/encode2.cuh +++ b/src/cuda_zfp/encode2.cuh @@ -163,9 +163,9 @@ size_t encode2launch(uint2 dims, cudaEventSynchronize(stop); cudaStreamSynchronize(0); - float miliseconds = 0.f; - cudaEventElapsedTime(&miliseconds, start, stop); - float seconds = miliseconds / 1000.f; + float milliseconds = 0.f; + cudaEventElapsedTime(&milliseconds, start, stop); + float seconds = milliseconds / 1000.f; float mb = (float(dims.x * dims.y) * sizeof(Scalar)) / (1024.f * 1024.f *1024.f); float rate = mb / seconds; printf("Encode elapsed time: %.5f (s)\n", seconds); diff --git a/src/cuda_zfp/encode3.cuh b/src/cuda_zfp/encode3.cuh index 78d05bd77..1edee9e99 100644 --- a/src/cuda_zfp/encode3.cuh +++ b/src/cuda_zfp/encode3.cuh @@ -171,9 +171,9 @@ size_t encode3launch(uint3 dims, cudaEventSynchronize(stop); cudaStreamSynchronize(0); - float miliseconds = 0; - cudaEventElapsedTime(&miliseconds, start, stop); - float seconds = miliseconds / 1000.f; + float milliseconds = 0; + cudaEventElapsedTime(&milliseconds, start, stop); + float seconds = milliseconds / 1000.f; float rate = (float(dims.x * dims.y * dims.z) * sizeof(Scalar) ) / seconds; rate /= 1024.f; rate /= 1024.f; diff --git a/src/template/encode.c b/src/template/encode.c index 0d63bb1f3..06c30263e 100644 --- a/src/template/encode.c +++ b/src/template/encode.c @@ -240,7 +240,7 @@ _t1(encode_ints, UInt)(bitstream* restrict_ stream, uint maxbits, uint maxprec, { /* use fastest available encoder implementation */ if (with_maxbits(maxbits, maxprec, size)) { - /* rate contrained path: encode partial bit planes */ + /* rate constrained path: encode partial bit planes */ if (size <= 64) return _t1(encode_few_ints, UInt)(stream, maxbits, maxprec, data, size); /* 1D, 2D, 3D blocks */ else diff --git a/tests/src/endtoend/serialExecBase.c b/tests/src/endtoend/serialExecBase.c index 95dfab394..067957b11 100644 --- a/tests/src/endtoend/serialExecBase.c +++ b/tests/src/endtoend/serialExecBase.c @@ -94,7 +94,7 @@ isCompressedBitrateComparableToChosenRate(struct setupVars* bundle) zfp_field* field = bundle->field; zfp_stream* stream = bundle->stream; - // integer arithemetic allows exact comparison + // integer arithmetic allows exact comparison size_t compressedBytes = zfp_compress(stream, field); if (compressedBytes == 0) { printf("Compression failed\n"); From 5190cae2da6977cbd7fc0ab075c62329574dd33b Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 20 Jul 2022 12:30:20 -0700 Subject: [PATCH 081/277] Add zfp_field_blocks() --- docs/source/high-level-api.rst | 8 ++++++++ include/zfp.h | 6 ++++++ src/zfp.c | 16 +++++++++++----- 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index ae543275f..263310757 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -852,6 +852,14 @@ Array Metadata ---- +.. c:function:: size_t zfp_field_blocks(const zfp_field* field) + + Return total number of *d*-dimensional blocks (whether partial or whole) + spanning the array. Each whole block consists of |4powd| scalars. + Available since |zfp| |fieldrelease|. + +---- + .. c:function:: zfp_bool zfp_field_stride(const zfp_field* field, ptrdiff_t* stride) Return :code:`zfp_false` if the array is stored contiguously as diff --git a/include/zfp.h b/include/zfp.h index 089ab94b6..26cc70ca5 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -464,6 +464,12 @@ zfp_field_size_bytes( const zfp_field* field /* field metadata */ ); +/* field size in number of blocks */ +size_t /* total number of blocks */ +zfp_field_blocks( + const zfp_field* field /* field metadata */ +); + /* field strides per dimension */ zfp_bool /* true if array is not contiguous */ zfp_field_stride( diff --git a/src/zfp.c b/src/zfp.c index 81625ad7b..6866d8b2d 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -240,6 +240,16 @@ zfp_field_size_bytes(const zfp_field* field) return field_index_span(field, NULL, NULL) * zfp_type_size(field->type); } +size_t +zfp_field_blocks(const zfp_field* field) +{ + size_t bx = (MAX(field->nx, 1u) + 3) / 4; + size_t by = (MAX(field->ny, 1u) + 3) / 4; + size_t bz = (MAX(field->nz, 1u) + 3) / 4; + size_t bw = (MAX(field->nw, 1u) + 3) / 4; + return bx * by * bz * bw; +} + zfp_bool zfp_field_stride(const zfp_field* field, ptrdiff_t* stride) { @@ -697,11 +707,7 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) { zfp_bool reversible = is_reversible(zfp); uint dims = zfp_field_dimensionality(field); - size_t mx = (MAX(field->nx, 1u) + 3) / 4; - size_t my = (MAX(field->ny, 1u) + 3) / 4; - size_t mz = (MAX(field->nz, 1u) + 3) / 4; - size_t mw = (MAX(field->nw, 1u) + 3) / 4; - size_t blocks = mx * my * mz * mw; + size_t blocks = zfp_field_blocks(field); uint values = 1u << (2 * dims); uint maxbits = 0; From b9166ad1a1595fb88221c19ad39d347f2bc26867 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 21 Jul 2022 11:43:16 -0700 Subject: [PATCH 082/277] Add LDFLAGS to support -lomp on Mac --- Config | 1 + docs/source/installation.rst | 2 -- examples/Makefile | 4 ++-- tests/Makefile | 2 +- utils/Makefile | 2 +- 5 files changed, 5 insertions(+), 6 deletions(-) diff --git a/Config b/Config index 107a56d33..08866a1a7 100644 --- a/Config +++ b/Config @@ -16,6 +16,7 @@ FC = gfortran OPTFLAGS = -O3 FLAGS = $(OPTFLAGS) -fPIC -pedantic -Wall -Wextra +LDFLAGS = SOFLAGS = # OpenMP compiler options ----------------------------------------------------- diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 416f454ae..d11ed7136 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -192,8 +192,6 @@ Regardless of the settings below, |libzfp| will always be built. .. c:macro:: BUILD_SHARED_LIBS Build shared objects (:file:`.so`, :file:`.dylib`, or :file:`.dll` files). - On macOS, the :code:`SOFLAGS` line in the :file:`Config` file may have - to be uncommented when building with GNU make. CMake default: on. GNU make default: off. diff --git a/examples/Makefile b/examples/Makefile index 0c922b4cc..b58871378 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -11,8 +11,8 @@ TARGETS = $(BINDIR)/array\ $(BINDIR)/speed INCS = -I../include LIBS = -L../lib -lzfp -CLIBS = $(LIBS) -lm -CXXLIBS = $(LIBS) +CLIBS = $(LIBS) $(LDFLAGS) -lm +CXXLIBS = $(LIBS) $(LDFLAGS) # add cfp examples when BUILD_CFP is enabled ifneq ($(BUILD_CFP),0) diff --git a/tests/Makefile b/tests/Makefile index ef5cfe363..fd3b21484 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -3,7 +3,7 @@ include ../Config BINDIR = ../bin TARGETS = $(BINDIR)/testzfp $(BINDIR)/testviews INCS = -I../include -I../array -LIBS = -L../lib -lzfp +LIBS = -L../lib -lzfp $(LDFLAGS) all: $(TARGETS) diff --git a/utils/Makefile b/utils/Makefile index 84edc6e95..dc7ef3e95 100644 --- a/utils/Makefile +++ b/utils/Makefile @@ -2,7 +2,7 @@ include ../Config TARGET = ../bin/zfp INCS = -I../include -LIBS = -L../lib -lzfp -lm +LIBS = -L../lib -lzfp $(LDFLAGS) -lm all: $(TARGET) From 0447f7e569b964d78e2451b094b1831258efa838 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 21 Jul 2022 11:51:48 -0700 Subject: [PATCH 083/277] Make codec thread-safe --- array/gencodec.h | 59 +++---- array/zfp/store.h | 35 ++++- array/zfp/view1.h | 14 +- array/zfp/view2.h | 14 +- array/zfp/view3.h | 14 +- array/zfp/view4.h | 14 +- array/zfpcodec.h | 346 +++++++++++++++++++++++++++++++----------- docs/source/codec.inc | 12 ++ tests/testviews.cpp | 82 +++++++--- 9 files changed, 440 insertions(+), 150 deletions(-) diff --git a/array/gencodec.h b/array/gencodec.h index 7557c2388..97d9df6ae 100644 --- a/array/gencodec.h +++ b/array/gencodec.h @@ -129,6 +129,9 @@ class generic_base { return false; } + // set thread safety mode (not required by this codec) + void set_thread_safety(bool) {} + // byte size of codec data structure components indicated by mask size_t size_bytes(uint mask = ZFP_DATA_ALL) const { @@ -193,6 +196,13 @@ class generic1 : public generic_base<1, ExternalType, InternalType> { : encode_block(offset, block); } + // decode contiguous 1D block + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1) + : decode_block(offset, block); + } + // encode 1D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx) const { @@ -206,13 +216,6 @@ class generic1 : public generic_base<1, ExternalType, InternalType> { return block_size_bits; } - // decode contiguous 1D block - size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1) - : decode_block(offset, block); - } - // decode 1D block to strided storage size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx) const { @@ -244,6 +247,13 @@ class generic2 : public generic_base<2, ExternalType, InternalType> { : encode_block(offset, block); } + // decode contiguous 2D block + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4) + : decode_block(offset, block); + } + // encode 2D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const { @@ -260,13 +270,6 @@ class generic2 : public generic_base<2, ExternalType, InternalType> { return block_size_bits; } - // decode contiguous 2D block - size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1, 4) - : decode_block(offset, block); - } - // decode 2D block to strided storage size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy) const { @@ -301,6 +304,13 @@ class generic3 : public generic_base<3, ExternalType, InternalType> { : encode_block(offset, block); } + // decode contiguous 3D block + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) + : decode_block(offset, block); + } + // encode 3D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const { @@ -320,13 +330,6 @@ class generic3 : public generic_base<3, ExternalType, InternalType> { return block_size_bits; } - // decode contiguous 3D block - size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) - : decode_block(offset, block); - } - // decode 3D block to strided storage size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const { @@ -364,6 +367,13 @@ class generic4 : public generic_base<4, ExternalType, InternalType> { : encode_block(offset, block); } + // decode contiguous 4D block + size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) + : decode_block(offset, block); + } + // encode 4D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const { @@ -386,13 +396,6 @@ class generic4 : public generic_base<4, ExternalType, InternalType> { return block_size_bits; } - // decode contiguous 4D block - size_t decode_block(bitstream_offset offset, uint shape, ExternalType* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) - : decode_block(offset, block); - } - // decode 4D block to strided storage size_t decode_block_strided(bitstream_offset offset, uint shape, ExternalType* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const { diff --git a/array/zfp/store.h b/array/zfp/store.h index 4b7b6b545..66d503e9d 100644 --- a/array/zfp/store.h +++ b/array/zfp/store.h @@ -123,6 +123,30 @@ class BlockStore { } } + // increment private view reference count (for thread safety) + void reference() + { +#ifdef _OPENMP + #pragma omp critical(references) + { + references++; + codec.set_thread_safety(references > 1); + } +#endif + } + + // decrement private view reference count (for thread safety) + void unreference() + { +#ifdef _OPENMP + #pragma omp critical(references) + { + references--; + codec.set_thread_safety(references > 1); + } +#endif + } + // byte size of store data structure components indicated by mask virtual size_t size_bytes(uint mask = ZFP_DATA_ALL) const { @@ -147,6 +171,7 @@ class BlockStore { BlockStore() : data(0), bytes(0), + references(0), index(0) {} @@ -175,6 +200,7 @@ class BlockStore { free(); zfp::internal::clone_aligned(data, s.data, s.bytes, ZFP_MEMORY_ALIGNMENT); bytes = s.bytes; + references = s.references; index = s.index; codec = s.codec; codec.open(data, bytes); @@ -216,10 +242,11 @@ class BlockStore { return static_cast(m); } - void* data; // pointer to compressed blocks - size_t bytes; // compressed data size - Index index; // block index (size and offset) - Codec codec; // compression codec + void* data; // pointer to compressed blocks + size_t bytes; // compressed data size + size_t references; // private view references to array (for thread safety) + Index index; // block index (size and offset) + Codec codec; // compression codec }; } // internal diff --git a/array/zfp/view1.h b/array/zfp/view1.h index 635f30077..006c6c2ec 100644 --- a/array/zfp/view1.h +++ b/array/zfp/view1.h @@ -165,11 +165,21 @@ class private_const_view : public preview { private_const_view(container_type* array, size_t cache_size = 0) : preview(array), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } private_const_view(container_type* array, size_t x, size_t nx, size_t cache_size = 0) : preview(array, x, nx), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } + + // destructor + ~private_const_view() + { + array->store.unreference(); + } // dimensions of (sub)array size_t size_x() const { return nx; } diff --git a/array/zfp/view2.h b/array/zfp/view2.h index 5ffddf62c..0049942af 100644 --- a/array/zfp/view2.h +++ b/array/zfp/view2.h @@ -348,11 +348,21 @@ class private_const_view : public preview { private_const_view(container_type* array, size_t cache_size = 0) : preview(array), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } private_const_view(container_type* array, size_t x, size_t y, size_t nx, size_t ny, size_t cache_size = 0) : preview(array, x, y, nx, ny), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } + + // destructor + ~private_const_view() + { + array->store.unreference(); + } // dimensions of (sub)array size_t size_x() const { return nx; } diff --git a/array/zfp/view3.h b/array/zfp/view3.h index c41808a7e..744ae80f7 100644 --- a/array/zfp/view3.h +++ b/array/zfp/view3.h @@ -423,11 +423,21 @@ class private_const_view : public preview { private_const_view(container_type* array, size_t cache_size = 0) : preview(array), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } private_const_view(container_type* array, size_t x, size_t y, size_t z, size_t nx, size_t ny, size_t nz, size_t cache_size = 0) : preview(array, x, y, z, nx, ny, nz), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } + + // destructor + ~private_const_view() + { + array->store.unreference(); + } // dimensions of (sub)array size_t size_x() const { return nx; } diff --git a/array/zfp/view4.h b/array/zfp/view4.h index 808e1e605..b4fb5b7bc 100644 --- a/array/zfp/view4.h +++ b/array/zfp/view4.h @@ -503,11 +503,21 @@ class private_const_view : public preview { private_const_view(container_type* array, size_t cache_size = 0) : preview(array), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } private_const_view(container_type* array, size_t x, size_t y, size_t z, size_t w, size_t nx, size_t ny, size_t nz, size_t nw, size_t cache_size = 0) : preview(array, x, y, z, w, nx, ny, nz, nw), cache(array->store, cache_size ? cache_size : array->cache.size()) - {} + { + array->store.reference(); + } + + // destructor + ~private_const_view() + { + array->store.unreference(); + } // dimensions of (sub)array size_t size_x() const { return nx; } diff --git a/array/zfpcodec.h b/array/zfpcodec.h index 989de59a6..9e470b099 100644 --- a/array/zfpcodec.h +++ b/array/zfpcodec.h @@ -18,7 +18,8 @@ class zfp_base { protected: // default constructor zfp_base() : - stream(zfp_stream_open(0)) + stream(zfp_stream_open(0)), + thread_safety(false) {} // destructor @@ -98,6 +99,9 @@ class zfp_base { // set expert mode parameters bool set_params(uint minbits, uint maxbits, uint maxprec, int maxexp) { return zfp_stream_set_params(stream, minbits, maxbits, maxprec, maxexp) == zfp_true; } + // set thread safety mode + void set_thread_safety(bool safety) { thread_safety = safety; } + // byte size of codec data structure components indicated by mask size_t size_bytes(uint mask = ZFP_DATA_ALL) const { @@ -124,27 +128,65 @@ class zfp_base { stream = zfp_stream_open(0); *stream = *codec.stream; stream->stream = 0; + thread_safety = codec.thread_safety; + } + + // make a thread-local copy of zfp stream and bit stream + zfp_stream clone_stream() const + { + zfp_stream zfp = *stream; + zfp.stream = stream_clone(zfp.stream); + return zfp; } // encode full contiguous block size_t encode_block(bitstream_offset offset, const Scalar* block) const { - stream_wseek(stream->stream, offset); - size_t size = zfp::encode_block(stream, block); - zfp_stream_flush(stream); - return size; + if (thread_safety) { + // make a thread-local copy of zfp stream and bit stream + zfp_stream zfp = clone_stream(); + size_t size = encode_block(&zfp, offset, block); + stream_close(zfp.stream); + return size; + } + else + return encode_block(stream, offset, block); } // decode full contiguous block size_t decode_block(bitstream_offset offset, Scalar* block) const { - stream_rseek(stream->stream, offset); - size_t size = zfp::decode_block(stream, block); - zfp_stream_align(stream); + if (thread_safety) { + // make a thread-local copy of zfp stream and bit stream + zfp_stream zfp = clone_stream(); + size_t size = decode_block(&zfp, offset, block); + stream_close(zfp.stream); + return size; + } + else + return decode_block(stream, offset, block); + } + + // encode full contiguous block + static size_t encode_block(zfp_stream* zfp, bitstream_offset offset, const Scalar* block) + { + stream_wseek(zfp->stream, offset); + size_t size = zfp::encode_block(zfp, block); + stream_flush(zfp->stream); + return size; + } + + // decode full contiguous block + static size_t decode_block(zfp_stream* zfp, bitstream_offset offset, Scalar* block) + { + stream_rseek(zfp->stream, offset); + size_t size = zfp::decode_block(zfp, block); + stream_align(zfp->stream); return size; } zfp_stream* stream; // compressed zfp stream + bool thread_safety; // thread safety state }; // 1D codec @@ -158,47 +200,77 @@ class zfp1 : public zfp_base<1, Scalar> { : encode_block(offset, block); } + // decode contiguous 1D block + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1) + : decode_block(offset, block); + } + // encode 1D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = encode_block_strided(&zfp, offset, shape, p, sx); + stream_close(zfp.stream); + return size; + } + else + return encode_block_strided(stream, offset, shape, p, sx); + } + + // decode 1D block to strided storage + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = decode_block_strided(&zfp, offset, shape, p, sx); + stream_close(zfp.stream); + return size; + } + else + return decode_block_strided(stream, offset, shape, p, sx); + } + +protected: + using zfp_base<1, Scalar>::clone_stream; + using zfp_base<1, Scalar>::encode_block; + using zfp_base<1, Scalar>::decode_block; + using zfp_base<1, Scalar>::stream; + using zfp_base<1, Scalar>::thread_safety; + + // encode 1D block from strided storage + static size_t encode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx) { size_t size; - stream_wseek(stream->stream, offset); + stream_wseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(stream, p, nx, sx); + size = zfp::encode_partial_block_strided(zfp, p, nx, sx); } else - size = zfp::encode_block_strided(stream, p, sx); - zfp_stream_flush(stream); + size = zfp::encode_block_strided(zfp, p, sx); + stream_flush(zfp->stream); return size; } - // decode contiguous 1D block - size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1) - : decode_block(offset, block); - } - // decode 1D block to strided storage - size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx) const + static size_t decode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx) { size_t size; - stream_rseek(stream->stream, offset); + stream_rseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(stream, p, nx, sx); + size = zfp::decode_partial_block_strided(zfp, p, nx, sx); } else - size = zfp::decode_block_strided(stream, p, sx); - zfp_stream_align(stream); + size = zfp::decode_block_strided(zfp, p, sx); + stream_align(zfp->stream); return size; } - -protected: - using zfp_base<1, Scalar>::encode_block; - using zfp_base<1, Scalar>::decode_block; - using zfp_base<1, Scalar>::stream; }; // 2D codec @@ -212,49 +284,79 @@ class zfp2 : public zfp_base<2, Scalar> { : encode_block(offset, block); } + // decode contiguous 2D block + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4) + : decode_block(offset, block); + } + // encode 2D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = encode_block_strided(&zfp, offset, shape, p, sx, sy); + stream_close(zfp.stream); + return size; + } + else + return encode_block_strided(stream, offset, shape, p, sx, sy); + } + + // decode 2D block to strided storage + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = decode_block_strided(&zfp, offset, shape, p, sx, sy); + stream_close(zfp.stream); + return size; + } + else + return decode_block_strided(stream, offset, shape, p, sx, sy); + } + +protected: + using zfp_base<2, Scalar>::clone_stream; + using zfp_base<2, Scalar>::encode_block; + using zfp_base<2, Scalar>::decode_block; + using zfp_base<2, Scalar>::stream; + using zfp_base<2, Scalar>::thread_safety; + + // encode 2D block from strided storage + static size_t encode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy) { size_t size; - stream_wseek(stream->stream, offset); + stream_wseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; uint ny = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(stream, p, nx, ny, sx, sy); + size = zfp::encode_partial_block_strided(zfp, p, nx, ny, sx, sy); } else - size = zfp::encode_block_strided(stream, p, sx, sy); - zfp_stream_flush(stream); + size = zfp::encode_block_strided(zfp, p, sx, sy); + stream_flush(zfp->stream); return size; } - // decode contiguous 2D block - size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1, 4) - : decode_block(offset, block); - } - // decode 2D block to strided storage - size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) const + static size_t decode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy) { size_t size; - stream_rseek(stream->stream, offset); + stream_rseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; uint ny = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(stream, p, nx, ny, sx, sy); + size = zfp::decode_partial_block_strided(zfp, p, nx, ny, sx, sy); } else - size = zfp::decode_block_strided(stream, p, sx, sy); - zfp_stream_align(stream); + size = zfp::decode_block_strided(zfp, p, sx, sy); + stream_align(zfp->stream); return size; } - -protected: - using zfp_base<2, Scalar>::encode_block; - using zfp_base<2, Scalar>::decode_block; - using zfp_base<2, Scalar>::stream; }; // 3D codec @@ -268,51 +370,81 @@ class zfp3 : public zfp_base<3, Scalar> { : encode_block(offset, block); } + // decode contiguous 3D block + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) + : decode_block(offset, block); + } + // encode 3D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = encode_block_strided(&zfp, offset, shape, p, sx, sy, sz); + stream_close(zfp.stream); + return size; + } + else + return encode_block_strided(stream, offset, shape, p, sx, sy, sz); + } + + // decode 3D block to strided storage + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = decode_block_strided(&zfp, offset, shape, p, sx, sy, sz); + stream_close(zfp.stream); + return size; + } + else + return decode_block_strided(stream, offset, shape, p, sx, sy, sz); + } + +protected: + using zfp_base<3, Scalar>::clone_stream; + using zfp_base<3, Scalar>::encode_block; + using zfp_base<3, Scalar>::decode_block; + using zfp_base<3, Scalar>::stream; + using zfp_base<3, Scalar>::thread_safety; + + // encode 3D block from strided storage + static size_t encode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) { size_t size; - stream_wseek(stream->stream, offset); + stream_wseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; uint ny = 4 - (shape & 3u); shape >>= 2; uint nz = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(stream, p, nx, ny, nz, sx, sy, sz); + size = zfp::encode_partial_block_strided(zfp, p, nx, ny, nz, sx, sy, sz); } else - size = zfp::encode_block_strided(stream, p, sx, sy, sz); - zfp_stream_flush(stream); + size = zfp::encode_block_strided(zfp, p, sx, sy, sz); + stream_flush(zfp->stream); return size; } - // decode contiguous 3D block - size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1, 4, 16) - : decode_block(offset, block); - } - // decode 3D block to strided storage - size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) const + static size_t decode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz) { size_t size; - stream_rseek(stream->stream, offset); + stream_rseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; uint ny = 4 - (shape & 3u); shape >>= 2; uint nz = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(stream, p, nx, ny, nz, sx, sy, sz); + size = zfp::decode_partial_block_strided(zfp, p, nx, ny, nz, sx, sy, sz); } else - size = zfp::decode_block_strided(stream, p, sx, sy, sz); - zfp_stream_align(stream); + size = zfp::decode_block_strided(zfp, p, sx, sy, sz); + stream_align(zfp->stream); return size; } - -protected: - using zfp_base<3, Scalar>::encode_block; - using zfp_base<3, Scalar>::decode_block; - using zfp_base<3, Scalar>::stream; }; // 4D codec @@ -326,53 +458,83 @@ class zfp4 : public zfp_base<4, Scalar> { : encode_block(offset, block); } + // decode contiguous 4D block + size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const + { + return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) + : decode_block(offset, block); + } + // encode 4D block from strided storage size_t encode_block_strided(bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = encode_block_strided(&zfp, offset, shape, p, sx, sy, sz, sw); + stream_close(zfp.stream); + return size; + } + else + return encode_block_strided(stream, offset, shape, p, sx, sy, sz, sw); + } + + // decode 4D block to strided storage + size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + { + if (thread_safety) { + // thread-safe implementation + zfp_stream zfp = clone_stream(); + size_t size = decode_block_strided(&zfp, offset, shape, p, sx, sy, sz, sw); + stream_close(zfp.stream); + return size; + } + else + return decode_block_strided(stream, offset, shape, p, sx, sy, sz, sw); + } + +protected: + using zfp_base<4, Scalar>::clone_stream; + using zfp_base<4, Scalar>::encode_block; + using zfp_base<4, Scalar>::decode_block; + using zfp_base<4, Scalar>::stream; + using zfp_base<4, Scalar>::thread_safety; + + // encode 4D block from strided storage + static size_t encode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, const Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) { size_t size; - stream_wseek(stream->stream, offset); + stream_wseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; uint ny = 4 - (shape & 3u); shape >>= 2; uint nz = 4 - (shape & 3u); shape >>= 2; uint nw = 4 - (shape & 3u); shape >>= 2; - size = zfp::encode_partial_block_strided(stream, p, nx, ny, nz, nw, sx, sy, sz, sw); + size = zfp::encode_partial_block_strided(zfp, p, nx, ny, nz, nw, sx, sy, sz, sw); } else - size = zfp::encode_block_strided(stream, p, sx, sy, sz, sw); - zfp_stream_flush(stream); + size = zfp::encode_block_strided(zfp, p, sx, sy, sz, sw); + stream_flush(zfp->stream); return size; } - // decode contiguous 4D block - size_t decode_block(bitstream_offset offset, uint shape, Scalar* block) const - { - return shape ? decode_block_strided(offset, shape, block, 1, 4, 16, 64) - : decode_block(offset, block); - } - // decode 4D block to strided storage - size_t decode_block_strided(bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) const + static size_t decode_block_strided(zfp_stream* zfp, bitstream_offset offset, uint shape, Scalar* p, ptrdiff_t sx, ptrdiff_t sy, ptrdiff_t sz, ptrdiff_t sw) { size_t size; - stream_rseek(stream->stream, offset); + stream_rseek(zfp->stream, offset); if (shape) { uint nx = 4 - (shape & 3u); shape >>= 2; uint ny = 4 - (shape & 3u); shape >>= 2; uint nz = 4 - (shape & 3u); shape >>= 2; uint nw = 4 - (shape & 3u); shape >>= 2; - size = zfp::decode_partial_block_strided(stream, p, nx, ny, nz, nw, sx, sy, sz, sw); + size = zfp::decode_partial_block_strided(zfp, p, nx, ny, nz, nw, sx, sy, sz, sw); } else - size = zfp::decode_block_strided(stream, p, sx, sy, sz, sw); - zfp_stream_align(stream); + size = zfp::decode_block_strided(zfp, p, sx, sy, sz, sw); + stream_align(zfp->stream); return size; } - -protected: - using zfp_base<4, Scalar>::encode_block; - using zfp_base<4, Scalar>::decode_block; - using zfp_base<4, Scalar>::stream; }; } // codec diff --git a/docs/source/codec.inc b/docs/source/codec.inc index a84343dfd..c806a4923 100644 --- a/docs/source/codec.inc +++ b/docs/source/codec.inc @@ -154,6 +154,18 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- +.. cpp:function:: bool codec::set_thread_safety(bool safety) + + Enable or disable thread safety. This function is called whenever |zfp| + is built with OpenMP support and when the number of mutable or immutable + :ref:`private views ` of an array changes. When + two or more private views of an array are accessed by separate threads, + multiple blocks may be compressed or decompressed simultaneously. The + codec then has to take care that there are no race conditions on the data + structures (e.g., :c:type:`bitstream`) used for (de)compression. + +---- + .. cpp:function:: size_t codec::size_bytes(uint mask = ZFP_DATA_ALL) const Return storage size of components of codec data structure indicated by diff --git a/tests/testviews.cpp b/tests/testviews.cpp index 3827c5265..892019411 100644 --- a/tests/testviews.cpp +++ b/tests/testviews.cpp @@ -2,16 +2,19 @@ #include #include #include +#ifdef _OPENMP +#include +#endif #include "zfparray2.h" #include "zfparray3.h" #define EPSILON 1e-3 -// random integer in {begin, ..., end - 1} +// random integer in {begin, ..., end} static size_t rand(size_t begin, size_t end) { - return begin + size_t(rand()) % (end - begin); + return begin + size_t(rand()) % (end - begin + 1); } // ensure f and g are sufficiently close @@ -19,6 +22,9 @@ static void verify(double f, double g) { if (std::fabs(f - g) > EPSILON) { +#ifdef _OPENMP + #pragma omp critical +#endif std::cerr << "error: " << f << " != " << g << std::endl; exit(EXIT_FAILURE); } @@ -33,9 +39,9 @@ usage() int main(int argc, char* argv[]) { - size_t nx = 16; - size_t ny = 16; - size_t nz = 16; + size_t nx = 8; + size_t ny = 48; + size_t nz = 32; size_t x0, y0, z0; size_t mx, my, mz; double rate = 16; @@ -65,12 +71,12 @@ int main(int argc, char* argv[]) if (argc < 10) { // generate random view - x0 = rand(0, nx); - y0 = rand(0, ny); - z0 = rand(0, nz); - mx = rand(0, nx - x0); - my = rand(0, ny - y0); - mz = rand(0, nz - z0); + x0 = rand(0, nx - 1); + y0 = rand(0, ny - 1); + z0 = rand(0, nz - 1); + mx = rand(1, nx - x0); + my = rand(1, ny - y0); + mz = rand(1, nz - z0); } // validate arguments @@ -112,9 +118,9 @@ int main(int argc, char* argv[]) // nested view of all of a std::cout << std::endl << "3D nested view" << std::endl; zfp::array3::nested_view nv(&a); - for (size_t z = 0; z < v.size_z(); z++) - for (size_t y = 0; y < v.size_y(); y++) - for (size_t x = 0; x < v.size_x(); x++) { + for (size_t z = 0; z < nv.size_z(); z++) + for (size_t y = 0; y < nv.size_y(); y++) + for (size_t x = 0; x < nv.size_x(); x++) { std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << nv[z][y][x] << std::endl; verify(a(x, y, z), nv[z][y][x]); } @@ -128,6 +134,8 @@ int main(int argc, char* argv[]) size_t x = it.i(); size_t y = it.j(); size_t z = it.k(); +std::cout << x << " " << y << " " << z << std::endl; +std::cout << mx << " " << my << " " << std::endl; verify(*it, p[x + mx * (y + my * z)]); } @@ -145,7 +153,7 @@ int main(int argc, char* argv[]) // 2D slice of a std::cout << std::endl << "2D slice" << std::endl; - size_t z = rand(0, nv.size_z()); + size_t z = rand(0, nv.size_z() - 1); zfp::array3::nested_view2 slice2(nv[z]); for (size_t y = 0; y < slice2.size_y(); y++) for (size_t x = 0; x < slice2.size_x(); x++) { @@ -164,7 +172,7 @@ int main(int argc, char* argv[]) // 1D slice of a std::cout << std::endl << "1D slice" << std::endl; - size_t y = rand(0, slice2.size_y()); + size_t y = rand(0, slice2.size_y() - 1); zfp::array3::nested_view1 slice1 = slice2[y]; for (size_t x = 0; x < slice1.size_x(); x++) { std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << slice1[x] << std::endl; @@ -180,8 +188,8 @@ int main(int argc, char* argv[]) verify(c(x, y), slice2[y][x]); } - // 2D thread-safe view of c - std::cout << std::endl << "2D private view" << std::endl; + // 2D thread-safe read-only view of c + std::cout << std::endl << "2D private read-only view" << std::endl; zfp::array2::private_const_view d(&c); for (size_t y = 0; y < c.size_y(); y++) for (size_t x = 0; x < c.size_x(); x++) { @@ -189,6 +197,44 @@ int main(int argc, char* argv[]) verify(c(x, y), d(x, y)); } +#ifdef _OPENMP + std::cout << std::endl << "multithreaded 2D private read-only views" << std::endl; + // copy c for verification; direct accesses to c are not thread-safe + double* data = new double[c.size()]; + c.get(data); + #pragma omp parallel + { + // make a thread-local view into c + zfp::array2::private_const_view d(&c); + for (size_t y = 0; y < d.size_y(); y++) + for (size_t x = 0; x < d.size_x(); x++) { + double val = data[x + nx * y]; + if (omp_get_thread_num() == 0) + std::cout << x << " " << y << ": " << val << " " << d(x, y) << std::endl; + verify(val, d(x, y)); + } + } + + std::cout << std::endl << "multithreaded 2D private read-write views" << std::endl; + #pragma omp parallel + { + // partition c into disjoint views + zfp::array2::private_view d(&c); + d.partition(omp_get_thread_num(), omp_get_num_threads()); + for (size_t j = 0; j < d.size_y(); j++) + for (size_t i = 0; i < d.size_x(); i++) { + d(i, j) += 1; + size_t x = d.global_x(i); + size_t y = d.global_y(j); + double val = data[x + nx * y] + 1; + if (omp_get_thread_num() == 0) + std::cout << x << " " << y << ": " << val << " " << d(i, j) << std::endl; + verify(val, d(i, j)); + } + } + delete[] data; +#endif + std::cout << std::endl << "all tests passed" << std::endl; return 0; From e89d803835a8dac7963ed17a89f39aac068930a4 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 21 Jul 2022 11:53:37 -0700 Subject: [PATCH 084/277] Handle empty field in zfp_field_blocks() --- src/zfp.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/src/zfp.c b/src/zfp.c index 6866d8b2d..f4cf27ece 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -243,11 +243,17 @@ zfp_field_size_bytes(const zfp_field* field) size_t zfp_field_blocks(const zfp_field* field) { - size_t bx = (MAX(field->nx, 1u) + 3) / 4; - size_t by = (MAX(field->ny, 1u) + 3) / 4; - size_t bz = (MAX(field->nz, 1u) + 3) / 4; - size_t bw = (MAX(field->nw, 1u) + 3) / 4; - return bx * by * bz * bw; + size_t bx = (field->nx + 3) / 4; + size_t by = (field->ny + 3) / 4; + size_t bz = (field->nz + 3) / 4; + size_t bw = (field->nw + 3) / 4; + switch (zfp_field_dimensionality(field)) { + case 1: return bx; + case 2: return bx * by; + case 3: return bx * by * bz; + case 4: return bx * by * bz * bw; + default: return 0; + } } zfp_bool From 11f9928499979ec5d6ccaf65542ccf3eb4807228 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 21 Jul 2022 15:51:29 -0700 Subject: [PATCH 085/277] Enabled OpenMP with testviews --- tests/CMakeLists.txt | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 2510e2d5d..7d0ded622 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -141,7 +141,11 @@ endif() # testviews add_executable(testviews testviews.cpp) -target_link_libraries(testviews zfp) +if(ZFP_WITH_OPENMP) + target_link_libraries(testviews zfp OpenMP::OpenMP_C) +else() + target_link_libraries(testviews zfp) +endif() target_compile_definitions(testviews PRIVATE ${zfp_compressed_array_defs}) add_test(NAME testviews COMMAND testviews) From 5c424bbff0a5abb50230a2245ee16a012c4b5921 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 22 Jul 2022 11:53:17 -0700 Subject: [PATCH 086/277] Update CHANGELOG [skip ci] --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f4b54a81b..a37077cb0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,6 +76,7 @@ few of the API changes, other than to cfp, should impact existing code. - #126: `make install` does not install Fortran module. - #127: Reversible mode reports incorrect compressed block size. - #150: cmocka tests do not build on macOS. +- #154: Thread safety is broken in `private_view` and `private_const_view`. - `ZFP_MAX_BITS` is off by one. - `diffusionC`, `iteratorC` are not being built with `gmake`. From 20a2efdcb41c12edd402a4698b500fdc4b53d267 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 22 Jul 2022 13:10:43 -0700 Subject: [PATCH 087/277] Use zfp_field_blocks() in codec --- array/gencodec.h | 11 +---------- array/zfpcodec.h | 6 +----- 2 files changed, 2 insertions(+), 15 deletions(-) diff --git a/array/gencodec.h b/array/gencodec.h index 7557c2388..b76a75e03 100644 --- a/array/gencodec.h +++ b/array/gencodec.h @@ -41,16 +41,7 @@ class generic_base { // conservative buffer size for current codec settings size_t buffer_size(const zfp_field* field) const { - // empty field case - if (!field->nx && !field->ny && !field->nz && !field->nw) - return 0; - // count number of blocks spanned by field - size_t bx = (std::max(field->nx, size_t(1)) + 3) / 4; - size_t by = (std::max(field->ny, size_t(1)) + 3) / 4; - size_t bz = (std::max(field->nz, size_t(1)) + 3) / 4; - size_t bw = (std::max(field->nw, size_t(1)) + 3) / 4; - size_t blocks = bx * by * bz * bw; - return blocks * block_size * sizeof(InternalType); + return zfp_field_blocks(field) * block_size * sizeof(InternalType); } // open diff --git a/array/zfpcodec.h b/array/zfpcodec.h index 989de59a6..125d5bd73 100644 --- a/array/zfpcodec.h +++ b/array/zfpcodec.h @@ -47,11 +47,7 @@ class zfp_base { if (zfp_stream_compression_mode(stream) != zfp_mode_fixed_rate) return zfp_stream_maximum_size(stream, field); // fixed-rate case: exclude header - size_t bx = (std::max(field->nx, size_t(1)) + 3) / 4; - size_t by = (std::max(field->ny, size_t(1)) + 3) / 4; - size_t bz = (std::max(field->nz, size_t(1)) + 3) / 4; - size_t bw = (std::max(field->nw, size_t(1)) + 3) / 4; - size_t blocks = bx * by * bz * bw; + size_t blocks = zfp_field_blocks(field); return zfp::internal::round_up(blocks * stream->maxbits, stream_alignment()) / CHAR_BIT; } From 86ffc4f201a61e68848e1bbf3092bc8dd7da7c60 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 22 Jul 2022 13:31:31 -0700 Subject: [PATCH 088/277] Rename ZFP_VERSION_DEVELOP, remove ZFP_VERSION_RELEASE --- CHANGELOG.md | 1 + include/zfp/version.h | 14 ++++++-------- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a37077cb0..eca10cbfc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,7 @@ few of the API changes, other than to cfp, should impact existing code. - `zfp::array::get_header()` has been replaced with a `zfp::array::header` constructor that accepts an array object. +- `ZFP_VERSION_RELEASE` is no longer defined (use `ZFP_VERSION_PATCH`). ### Fixed diff --git a/include/zfp/version.h b/include/zfp/version.h index 784f1cf18..a337ff692 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -2,19 +2,17 @@ #define ZFP_VERSION_H /* library version information */ -#define ZFP_VERSION_MAJOR 0 /* library major version number */ -#define ZFP_VERSION_MINOR 5 /* library minor version number */ -#define ZFP_VERSION_PATCH 5 /* library patch version number */ -#define ZFP_VERSION_TWEAK 0 /* library tweak version number */ +#define ZFP_VERSION_MAJOR 0 /* library major version number */ +#define ZFP_VERSION_MINOR 5 /* library minor version number */ +#define ZFP_VERSION_PATCH 5 /* library patch version number */ +#define ZFP_VERSION_TWEAK 0 /* library tweak version number */ -#define ZFP_VERSION_RELEASE ZFP_VERSION_PATCH +/* defined for work in progress (indicates unofficial release) */ +#define ZFP_VERSION_DEVELOP 1 /* codec version number (see also zfp_codec_version) */ #define ZFP_CODEC 5 -/* whether this is a full release or intermediate version */ -#define ZFP_DEVELOP 1 - /* stringification */ #define _zfp_str_(x) # x #define _zfp_str(x) _zfp_str_(x) From 736581dd4b44a18aaaaf2dcee30b446074077dcd Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 24 Jul 2022 09:40:38 -0700 Subject: [PATCH 089/277] Silence implicit conversion warnings --- array/zfp/iterator1.h | 4 ++-- array/zfp/iterator2.h | 8 ++++---- array/zfp/iterator3.h | 12 ++++++------ array/zfp/iterator4.h | 16 ++++++++-------- array/zfp/pointer1.h | 2 +- array/zfp/pointer2.h | 4 ++-- array/zfp/pointer3.h | 6 +++--- array/zfp/pointer4.h | 8 ++++---- examples/inplace.c | 2 +- src/template/decode.c | 4 ++-- src/template/decodef.c | 5 +++-- src/template/decodei.c | 2 +- src/template/encode.c | 4 ++-- src/template/encodef.c | 6 +++--- src/template/encodei.c | 2 +- src/template/revdecode.c | 6 +++--- src/template/revdecodef.c | 4 ++-- src/template/revencode.c | 6 +++--- src/template/revencodef.c | 4 ++-- src/zfp.c | 10 +++++----- 20 files changed, 58 insertions(+), 57 deletions(-) diff --git a/array/zfp/iterator1.h b/array/zfp/iterator1.h index f3eb206d2..ff40f1178 100644 --- a/array/zfp/iterator1.h +++ b/array/zfp/iterator1.h @@ -58,10 +58,10 @@ class const_iterator : public const_handle { protected: // sequential offset associated with index x plus delta d - difference_type offset(difference_type d = 0) const { return static_cast(x - container->min_x() + d); } + difference_type offset(difference_type d = 0) const { return static_cast(x - container->min_x() + size_t(d)); } // index x associated with sequential offset p - void index(size_t& x, difference_type p) const { x = container->min_x() + p; } + void index(size_t& x, difference_type p) const { x = container->min_x() + size_t(p); } // advance iterator by d void advance(difference_type d) { index(x, offset(d)); } diff --git a/array/zfp/iterator2.h b/array/zfp/iterator2.h index bf0f674f7..f5c22685c 100644 --- a/array/zfp/iterator2.h +++ b/array/zfp/iterator2.h @@ -95,10 +95,10 @@ class const_iterator : public const_handle { } else { size_t m = ~size_t(3); - size_t by = std::max((ymin + p / nx) & m, ymin); size_t sy = std::min((by + 4) & m, ymax) - by; p -= (by - ymin) * nx; - size_t bx = std::max((xmin + p / sy) & m, xmin); size_t sx = std::min((bx + 4) & m, xmax) - bx; p -= (bx - xmin) * sy; - y = by + p / sx; p -= (y - by) * sx; - x = bx + p; p -= (x - bx); + size_t by = std::max((ymin + size_t(p / ptrdiff_t(nx))) & m, ymin); size_t sy = std::min((by + 4) & m, ymax) - by; p -= (by - ymin) * nx; + size_t bx = std::max((xmin + size_t(p / ptrdiff_t(sy))) & m, xmin); size_t sx = std::min((bx + 4) & m, xmax) - bx; p -= (bx - xmin) * sy; + y = by + size_t(p / ptrdiff_t(sx)); p -= (y - by) * sx; + x = bx + size_t(p); p -= (x - bx); } } diff --git a/array/zfp/iterator3.h b/array/zfp/iterator3.h index 386ef26cf..30ac7f50d 100644 --- a/array/zfp/iterator3.h +++ b/array/zfp/iterator3.h @@ -105,12 +105,12 @@ class const_iterator : public const_handle { } else { size_t m = ~size_t(3); - size_t bz = std::max((zmin + p / (nx * ny)) & m, zmin); size_t sz = std::min((bz + 4) & m, zmax) - bz; p -= (bz - zmin) * nx * ny; - size_t by = std::max((ymin + p / (nx * sz)) & m, ymin); size_t sy = std::min((by + 4) & m, ymax) - by; p -= (by - ymin) * nx * sz; - size_t bx = std::max((xmin + p / (sy * sz)) & m, xmin); size_t sx = std::min((bx + 4) & m, xmax) - bx; p -= (bx - xmin) * sy * sz; - z = bz + p / (sx * sy); p -= (z - bz) * sx * sy; - y = by + p / sx; p -= (y - by) * sx; - x = bx + p; p -= (x - bx); + size_t bz = std::max((zmin + size_t(p / ptrdiff_t(nx * ny))) & m, zmin); size_t sz = std::min((bz + 4) & m, zmax) - bz; p -= (bz - zmin) * nx * ny; + size_t by = std::max((ymin + size_t(p / ptrdiff_t(nx * sz))) & m, ymin); size_t sy = std::min((by + 4) & m, ymax) - by; p -= (by - ymin) * nx * sz; + size_t bx = std::max((xmin + size_t(p / ptrdiff_t(sy * sz))) & m, xmin); size_t sx = std::min((bx + 4) & m, xmax) - bx; p -= (bx - xmin) * sy * sz; + z = bz + size_t(p / ptrdiff_t(sx * sy)); p -= (z - bz) * sx * sy; + y = by + size_t(p / ptrdiff_t(sx)); p -= (y - by) * sx; + x = bx + size_t(p); p -= (x - bx); } } diff --git a/array/zfp/iterator4.h b/array/zfp/iterator4.h index d4530e42e..2f28c9d3a 100644 --- a/array/zfp/iterator4.h +++ b/array/zfp/iterator4.h @@ -115,14 +115,14 @@ class const_iterator : public const_handle { } else { size_t m = ~size_t(3); - size_t bw = std::max((wmin + p / (nx * ny * nz)) & m, wmin); size_t sw = std::min((bw + 4) & m, wmax) - bw; p -= (bw - wmin) * nx * ny * nz; - size_t bz = std::max((zmin + p / (nx * ny * sw)) & m, zmin); size_t sz = std::min((bz + 4) & m, zmax) - bz; p -= (bz - zmin) * nx * ny * sw; - size_t by = std::max((ymin + p / (nx * sz * sw)) & m, ymin); size_t sy = std::min((by + 4) & m, ymax) - by; p -= (by - ymin) * nx * sz * sw; - size_t bx = std::max((xmin + p / (sy * sz * sw)) & m, xmin); size_t sx = std::min((bx + 4) & m, xmax) - bx; p -= (bx - xmin) * sy * sz * sw; - w = bw + p / (sx * sy * sz); p -= (w - bw) * sx * sy * sz; - z = bz + p / (sx * sy); p -= (z - bz) * sx * sy; - y = by + p / sx; p -= (y - by) * sx; - x = bx + p; p -= (x - bx); + size_t bw = std::max((wmin + size_t(p / ptrdiff_t(nx * ny * nz))) & m, wmin); size_t sw = std::min((bw + 4) & m, wmax) - bw; p -= (bw - wmin) * nx * ny * nz; + size_t bz = std::max((zmin + size_t(p / ptrdiff_t(nx * ny * sw))) & m, zmin); size_t sz = std::min((bz + 4) & m, zmax) - bz; p -= (bz - zmin) * nx * ny * sw; + size_t by = std::max((ymin + size_t(p / ptrdiff_t(nx * sz * sw))) & m, ymin); size_t sy = std::min((by + 4) & m, ymax) - by; p -= (by - ymin) * nx * sz * sw; + size_t bx = std::max((xmin + size_t(p / ptrdiff_t(sy * sz * sw))) & m, xmin); size_t sx = std::min((bx + 4) & m, xmax) - bx; p -= (bx - xmin) * sy * sz * sw; + w = bw + size_t(p / ptrdiff_t(sx * sy * sz)); p -= (w - bw) * sx * sy * sz; + z = bz + size_t(p / ptrdiff_t(sx * sy)); p -= (z - bz) * sx * sy; + y = by + size_t(p / ptrdiff_t(sx)); p -= (y - by) * sx; + x = bx + size_t(p); p -= (x - bx); } } diff --git a/array/zfp/pointer1.h b/array/zfp/pointer1.h index 45fdd90c4..5d08962ab 100644 --- a/array/zfp/pointer1.h +++ b/array/zfp/pointer1.h @@ -50,7 +50,7 @@ class const_pointer : public const_handle { protected: ptrdiff_t offset(ptrdiff_t d = 0) const { return static_cast(x - container->min_x()) + d; } - void index(size_t& x, ptrdiff_t p) const { x = container->min_x() + p; } + void index(size_t& x, ptrdiff_t p) const { x = container->min_x() + size_t(p); } void advance(ptrdiff_t d) { index(x, offset(d)); } void increment() { ++x; } void decrement() { --x; } diff --git a/array/zfp/pointer2.h b/array/zfp/pointer2.h index d025b1464..92c107499 100644 --- a/array/zfp/pointer2.h +++ b/array/zfp/pointer2.h @@ -52,8 +52,8 @@ class const_pointer : public const_handle { ptrdiff_t offset(ptrdiff_t d = 0) const { return static_cast(x - container->min_x() + container->size_x() * (y - container->min_y())) + d; } void index(size_t& x, size_t& y, ptrdiff_t p) const { - x = container->min_x() + p % container->size_x(); p /= container->size_x(); - y = container->min_y() + p; + x = container->min_x() + size_t(p % ptrdiff_t(container->size_x())); p /= container->size_x(); + y = container->min_y() + size_t(p); } void advance(ptrdiff_t d) { index(x, y, offset(d)); } void increment() diff --git a/array/zfp/pointer3.h b/array/zfp/pointer3.h index 00e3ae8e5..a96026208 100644 --- a/array/zfp/pointer3.h +++ b/array/zfp/pointer3.h @@ -52,9 +52,9 @@ class const_pointer : public const_handle { ptrdiff_t offset(ptrdiff_t d = 0) const { return static_cast(x - container->min_x() + container->size_x() * (y - container->min_y() + container->size_y() * (z - container->min_z()))) + d; } void index(size_t& x, size_t& y, size_t& z, ptrdiff_t p) const { - x = container->min_x() + p % container->size_x(); p /= container->size_x(); - y = container->min_y() + p % container->size_y(); p /= container->size_y(); - z = container->min_z() + p; + x = container->min_x() + size_t(p % ptrdiff_t(container->size_x())); p /= container->size_x(); + y = container->min_y() + size_t(p % ptrdiff_t(container->size_y())); p /= container->size_y(); + z = container->min_z() + size_t(p); } void advance(ptrdiff_t d) { index(x, y, z, offset(d)); } void increment() diff --git a/array/zfp/pointer4.h b/array/zfp/pointer4.h index 5bbc2ba35..3f3eb2391 100644 --- a/array/zfp/pointer4.h +++ b/array/zfp/pointer4.h @@ -52,10 +52,10 @@ class const_pointer : public const_handle { ptrdiff_t offset(ptrdiff_t d = 0) const { return static_cast(x - container->min_x() + container->size_x() * (y - container->min_y() + container->size_y() * (z - container->min_z() + container->size_z() * (w - container->min_w())))) + d; } void index(size_t& x, size_t& y, size_t& z, size_t & w, ptrdiff_t p) const { - x = container->min_x() + p % container->size_x(); p /= container->size_x(); - y = container->min_y() + p % container->size_y(); p /= container->size_y(); - z = container->min_z() + p % container->size_z(); p /= container->size_z(); - w = container->min_w() + p; + x = container->min_x() + size_t(p % ptrdiff_t(container->size_x())); p /= container->size_x(); + y = container->min_y() + size_t(p % ptrdiff_t(container->size_y())); p /= container->size_y(); + z = container->min_z() + size_t(p % ptrdiff_t(container->size_z())); p /= container->size_z(); + w = container->min_w() + size_t(p); } void advance(ptrdiff_t d) { index(x, y, z, w, offset(d)); } void increment() diff --git a/examples/inplace.c b/examples/inplace.c index 3764166b5..9516240d0 100644 --- a/examples/inplace.c +++ b/examples/inplace.c @@ -46,7 +46,7 @@ process(double* buffer, uint blocks, double tolerance) ptr = buffer; for (i = 0; i < blocks; i++) { offset[i] = stream_wtell(stream); - bits = zfp_encode_block_double_2(zfp, ptr); + bits = (uint)zfp_encode_block_double_2(zfp, ptr); if (!bits) { fprintf(stderr, "compression failed\n"); return 0; diff --git a/src/template/decode.c b/src/template/decode.c index 2df1a958a..2f39bebd5 100644 --- a/src/template/decode.c +++ b/src/template/decode.c @@ -268,9 +268,9 @@ _t1(decode_ints, UInt)(bitstream* restrict_ stream, uint maxbits, uint maxprec, /* decode block of integers */ static uint -_t2(decode_block, Int, DIMS)(bitstream* stream, int minbits, int maxbits, int maxprec, Int* iblock) +_t2(decode_block, Int, DIMS)(bitstream* stream, uint minbits, uint maxbits, uint maxprec, Int* iblock) { - int bits; + uint bits; cache_align_(UInt ublock[BLOCK_SIZE]); /* decode integer coefficients */ bits = _t1(decode_ints, UInt)(stream, maxbits, maxprec, ublock, BLOCK_SIZE); diff --git a/src/template/decodef.c b/src/template/decodef.c index 3191c9469..e8fa40c81 100644 --- a/src/template/decodef.c +++ b/src/template/decodef.c @@ -10,13 +10,14 @@ _t2(decode_block, Scalar, DIMS)(zfp_stream* zfp, Scalar* fblock) /* test if block has nonzero values */ if (stream_read_bit(zfp->stream)) { cache_align_(Int iblock[BLOCK_SIZE]); - int emax, maxprec; + uint maxprec; + int emax; /* decode common exponent */ bits += EBITS; emax = (int)stream_read_bits(zfp->stream, EBITS) - EBIAS; maxprec = precision(emax, zfp->maxprec, zfp->minexp, DIMS); /* decode integer block */ - bits += _t2(decode_block, Int, DIMS)(zfp->stream, zfp->minbits - bits, zfp->maxbits - bits, maxprec, iblock); + bits += _t2(decode_block, Int, DIMS)(zfp->stream, zfp->minbits - MIN(bits, zfp->minbits), zfp->maxbits - bits, maxprec, iblock); /* perform inverse block-floating-point transform */ _t1(inv_cast, Scalar)(iblock, fblock, BLOCK_SIZE, emax); } diff --git a/src/template/decodei.c b/src/template/decodei.c index 8a35a8d8d..3cea9651f 100644 --- a/src/template/decodei.c +++ b/src/template/decodei.c @@ -1,4 +1,4 @@ -static uint _t2(rev_decode_block, Int, DIMS)(bitstream* stream, int minbits, int maxbits, Int* iblock); +static uint _t2(rev_decode_block, Int, DIMS)(bitstream* stream, uint minbits, uint maxbits, Int* iblock); /* public functions -------------------------------------------------------- */ diff --git a/src/template/encode.c b/src/template/encode.c index 06c30263e..c085a4ab2 100644 --- a/src/template/encode.c +++ b/src/template/encode.c @@ -257,9 +257,9 @@ _t1(encode_ints, UInt)(bitstream* restrict_ stream, uint maxbits, uint maxprec, /* encode block of integers */ static uint -_t2(encode_block, Int, DIMS)(bitstream* stream, int minbits, int maxbits, int maxprec, Int* iblock) +_t2(encode_block, Int, DIMS)(bitstream* stream, uint minbits, uint maxbits, uint maxprec, Int* iblock) { - int bits; + uint bits; cache_align_(UInt ublock[BLOCK_SIZE]); /* perform decorrelating transform */ _t2(fwd_xform, Int, DIMS)(iblock); diff --git a/src/template/encodef.c b/src/template/encodef.c index 86da10c75..10e504385 100644 --- a/src/template/encodef.c +++ b/src/template/encodef.c @@ -65,8 +65,8 @@ _t2(encode_block, Scalar, DIMS)(zfp_stream* zfp, const Scalar* fblock) uint bits = 1; /* compute maximum exponent */ int emax = _t1(exponent_block, Scalar)(fblock, BLOCK_SIZE); - int maxprec = precision(emax, zfp->maxprec, zfp->minexp, DIMS); - uint e = maxprec ? emax + EBIAS : 0; + uint maxprec = precision(emax, zfp->maxprec, zfp->minexp, DIMS); + uint e = maxprec ? (uint)(emax + EBIAS) : 0; /* encode block only if biased exponent is nonzero */ if (e) { cache_align_(Int iblock[BLOCK_SIZE]); @@ -76,7 +76,7 @@ _t2(encode_block, Scalar, DIMS)(zfp_stream* zfp, const Scalar* fblock) /* perform forward block-floating-point transform */ _t1(fwd_cast, Scalar)(iblock, fblock, BLOCK_SIZE, emax); /* encode integer block */ - bits += _t2(encode_block, Int, DIMS)(zfp->stream, zfp->minbits - bits, zfp->maxbits - bits, maxprec, iblock); + bits += _t2(encode_block, Int, DIMS)(zfp->stream, zfp->minbits - MIN(bits, zfp->minbits), zfp->maxbits - bits, maxprec, iblock); } else { /* write single zero-bit to indicate that all values are zero */ diff --git a/src/template/encodei.c b/src/template/encodei.c index 46b6e459b..2aa4e7e3d 100644 --- a/src/template/encodei.c +++ b/src/template/encodei.c @@ -1,4 +1,4 @@ -static uint _t2(rev_encode_block, Int, DIMS)(bitstream* stream, int minbits, int maxbits, int maxprec, Int* iblock); +static uint _t2(rev_encode_block, Int, DIMS)(bitstream* stream, uint minbits, uint maxbits, uint maxprec, Int* iblock); /* public functions -------------------------------------------------------- */ diff --git a/src/template/revdecode.c b/src/template/revdecode.c index 07205f58f..115b0a177 100644 --- a/src/template/revdecode.c +++ b/src/template/revdecode.c @@ -31,11 +31,11 @@ _t1(rev_inv_lift, Int)(Int* p, uint s) /* decode block of integers using reversible algorithm */ static uint -_t2(rev_decode_block, Int, DIMS)(bitstream* stream, int minbits, int maxbits, Int* iblock) +_t2(rev_decode_block, Int, DIMS)(bitstream* stream, uint minbits, uint maxbits, Int* iblock) { /* decode number of significant bits */ - int bits = PBITS; - int prec = (int)stream_read_bits(stream, PBITS) + 1; + uint bits = PBITS; + uint prec = (uint)stream_read_bits(stream, PBITS) + 1; cache_align_(UInt ublock[BLOCK_SIZE]); /* decode integer coefficients */ bits += _t1(decode_ints, UInt)(stream, maxbits - bits, prec, ublock, BLOCK_SIZE); diff --git a/src/template/revdecodef.c b/src/template/revdecodef.c index 6777ce018..5fafcefcf 100644 --- a/src/template/revdecodef.c +++ b/src/template/revdecodef.c @@ -30,7 +30,7 @@ _t2(rev_decode_block, Scalar, DIMS)(zfp_stream* zfp, Scalar* fblock) bits++; if (stream_read_bit(zfp->stream)) { /* decode integer block */ - bits += _t2(rev_decode_block, Int, DIMS)(zfp->stream, zfp->minbits - bits, zfp->maxbits - bits, iblock); + bits += _t2(rev_decode_block, Int, DIMS)(zfp->stream, zfp->minbits - MIN(bits, zfp->minbits), zfp->maxbits - bits, iblock); /* reinterpret integers as floating values */ _t1(rev_inv_reinterpret, Scalar)(iblock, fblock, BLOCK_SIZE); } @@ -40,7 +40,7 @@ _t2(rev_decode_block, Scalar, DIMS)(zfp_stream* zfp, Scalar* fblock) bits += EBITS; emax = (int)stream_read_bits(zfp->stream, EBITS) - EBIAS; /* decode integer block */ - bits += _t2(rev_decode_block, Int, DIMS)(zfp->stream, zfp->minbits - bits, zfp->maxbits - bits, iblock); + bits += _t2(rev_decode_block, Int, DIMS)(zfp->stream, zfp->minbits - MIN(bits, zfp->minbits), zfp->maxbits - bits, iblock); /* perform inverse block-floating-point transform */ _t1(rev_inv_cast, Scalar)(iblock, fblock, BLOCK_SIZE, emax); } diff --git a/src/template/revencode.c b/src/template/revencode.c index 3473cdd54..fa1621406 100644 --- a/src/template/revencode.c +++ b/src/template/revencode.c @@ -51,10 +51,10 @@ _t1(rev_precision, UInt)(const UInt* block, uint n) /* encode block of integers using reversible algorithm */ static uint -_t2(rev_encode_block, Int, DIMS)(bitstream* stream, int minbits, int maxbits, int maxprec, Int* iblock) +_t2(rev_encode_block, Int, DIMS)(bitstream* stream, uint minbits, uint maxbits, uint maxprec, Int* iblock) { - int bits = PBITS; - int prec; + uint bits = PBITS; + uint prec; cache_align_(UInt ublock[BLOCK_SIZE]); /* perform decorrelating transform */ _t2(rev_fwd_xform, Int, DIMS)(iblock); diff --git a/src/template/revencodef.c b/src/template/revencodef.c index 789f9b938..ee270aa77 100644 --- a/src/template/revencodef.c +++ b/src/template/revencodef.c @@ -53,7 +53,7 @@ _t2(rev_encode_block, Scalar, DIMS)(zfp_stream* zfp, const Scalar* fblock) /* test if block-floating-point transform is reversible */ if (_t1(rev_fwd_reversible, Scalar)(iblock, fblock, BLOCK_SIZE, emax)) { /* transform is reversible; test if block has any non-zeros */ - uint e = emax + EBIAS; + uint e = (uint)(emax + EBIAS); if (e) { /* encode common exponent */ bits += 2; @@ -75,6 +75,6 @@ _t2(rev_encode_block, Scalar, DIMS)(zfp_stream* zfp, const Scalar* fblock) stream_write_bits(zfp->stream, 3, 2); } /* losslessly encode integers */ - bits += _t2(rev_encode_block, Int, DIMS)(zfp->stream, zfp->minbits - bits, zfp->maxbits - bits, zfp->maxprec, iblock); + bits += _t2(rev_encode_block, Int, DIMS)(zfp->stream, zfp->minbits - MIN(bits, zfp->minbits), zfp->maxbits - bits, zfp->maxprec, iblock); return bits; } diff --git a/src/zfp.c b/src/zfp.c index f4cf27ece..844e3a984 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -35,7 +35,7 @@ field_index_span(const zfp_field* field, ptrdiff_t* min, ptrdiff_t* max) *min = imin; if (max) *max = imax; - return imax - imin + 1; + return (size_t)(imax - imin + 1); } static zfp_bool @@ -663,7 +663,7 @@ zfp_stream_mode(const zfp_stream* zfp) /* minexp is [ZFP_MIN_EXP=-1074, 843] */ /* returns [2177, ZFP_MODE_SHORT_MAX=4094] */ /* +1 because skipped 2176 */ - return (zfp->minexp - ZFP_MIN_EXP) + (2048 + 128 + 1); + return (uint64)(zfp->minexp - ZFP_MIN_EXP) + (2048 + 128 + 1); else break; @@ -679,7 +679,7 @@ zfp_stream_mode(const zfp_stream* zfp) minbits = MAX(1, MIN(zfp->minbits, 0x8000u)) - 1; maxbits = MAX(1, MIN(zfp->maxbits, 0x8000u)) - 1; maxprec = MAX(1, MIN(zfp->maxprec, 0x0080u)) - 1; - minexp = MAX(0, MIN(zfp->minexp + 16495, 0x7fff)); + minexp = (uint)MAX(0, MIN(zfp->minexp + 16495, 0x7fff)); mode <<= 15; mode += minexp; mode <<= 7; mode += maxprec; mode <<= 15; mode += maxbits; @@ -1092,7 +1092,7 @@ zfp_compress(zfp_stream* zfp, const zfp_field* field) #endif }; uint exec = zfp->exec.policy; - uint strided = zfp_field_stride(field, NULL); + uint strided = (uint)zfp_field_stride(field, NULL); uint dims = zfp_field_dimensionality(field); uint type = field->type; void (*compress)(zfp_stream*, const zfp_field*); @@ -1152,7 +1152,7 @@ zfp_decompress(zfp_stream* zfp, zfp_field* field) #endif }; uint exec = zfp->exec.policy; - uint strided = zfp_field_stride(field, NULL); + uint strided = (uint)zfp_field_stride(field, NULL); uint dims = zfp_field_dimensionality(field); uint type = field->type; void (*decompress)(zfp_stream*, zfp_field*); From f08994139f8be12edc740f15157ce45da4800395 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 25 Jul 2022 08:16:44 -0700 Subject: [PATCH 090/277] Clarify that low-level API ignores execution policy --- docs/source/execution.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/source/execution.rst b/docs/source/execution.rst index 81d066788..8059b0865 100644 --- a/docs/source/execution.rst +++ b/docs/source/execution.rst @@ -27,8 +27,11 @@ responsible for compressing a *chunk* of several contiguous blocks. This section describes the |zfp| parallel compression algorithm and explains how to configure |libzfp| and enable parallel compression at run time via -its :ref:`high-level C API `. Parallel compression is not supported -via the :ref:`low-level API `. +its :ref:`high-level C API `. + +.. note:: + Parallel compression is not supported via the :ref:`low-level API `, + which ignores all execution policy settings and always executes in serial. Execution Policies From 6c46dccf92f85cecbc8d0cf7f5f07cc8d6fd1581 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Mon, 25 Jul 2022 11:27:12 -0700 Subject: [PATCH 091/277] Update makefiles and fix a few inconsistencies --- Makefile | 4 ++-- cfp/Makefile | 8 ++++---- examples/Makefile | 19 +++++++++++++------ fortran/Makefile | 14 +++++++------- include/zfp/cfp.h | 4 ++-- include/zfp/codec/generic.hpp | 4 ++-- include/zfp/codec/zfpcodec.hpp | 2 +- include/zfp/constarray1.hpp | 4 ++-- include/zfp/constarray2.hpp | 4 ++-- include/zfp/constarray3.hpp | 4 ++-- include/zfp/constarray4.hpp | 4 ++-- include/zfp/factory.hpp | 2 +- include/zfp/internal/array/traits.hpp | 2 ++ .../internal/{array => codec}/genheader.hpp | 0 include/zfp/internal/zfp/inline.h | 4 ++-- tests/Makefile | 2 +- tests/fortran/testFortran.f | 2 +- tests/testzfp.cpp | 2 +- 18 files changed, 47 insertions(+), 38 deletions(-) rename include/zfp/internal/{array => codec}/genheader.hpp (100%) diff --git a/Makefile b/Makefile index bddc72ae1..db433ab23 100644 --- a/Makefile +++ b/Makefile @@ -9,7 +9,7 @@ all: @echo $(LIBRARY) @cd src; $(MAKE) clean $(LIBRARY) ifneq ($(BUILD_CFP),0) - @cd cfp/src; $(MAKE) clean $(LIBRARY) + @cd cfp; $(MAKE) clean $(LIBRARY) endif ifneq ($(BUILD_ZFORP),0) @cd fortran; $(MAKE) clean $(LIBRARY) @@ -33,7 +33,7 @@ test: # clean all clean: @cd src; $(MAKE) clean - @cd cfp/src; $(MAKE) clean + @cd src; $(MAKE) clean @cd fortran; $(MAKE) clean @cd utils; $(MAKE) clean @cd tests; $(MAKE) clean diff --git a/cfp/Makefile b/cfp/Makefile index 37cb00adf..37881a76c 100644 --- a/cfp/Makefile +++ b/cfp/Makefile @@ -1,9 +1,9 @@ -include ../../Config +include ../Config -LIBDIR = ../../lib +LIBDIR = ../lib TARGETS = $(LIBDIR)/libcfp.a $(LIBDIR)/libcfp.so -OBJECTS = cfparray.o -INCS = -I../include -I../../include -I../../array -I../../src +OBJECTS = cfp.o +INCS = -I../include -I../src static: $(LIBDIR)/libcfp.a diff --git a/examples/Makefile b/examples/Makefile index 0c922b4cc..82578e9d2 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -16,29 +16,36 @@ CXXLIBS = $(LIBS) # add cfp examples when BUILD_CFP is enabled ifneq ($(BUILD_CFP),0) - TARGETS += $(BINDIR)/diffusionC + TARGETS += $(BINDIR)/diffusionC\ + $(BINDIR)/iteratorC endif all: $(TARGETS) $(BINDIR)/array: array.cpp ../lib/$(LIBZFP) - $(CXX) $(CXXFLAGS) $(INCS) -I../array array.cpp $(CXXLIBS) -o $@ + $(CXX) $(CXXFLAGS) $(INCS) array.cpp $(CXXLIBS) -o $@ $(BINDIR)/diffusion: diffusion.cpp ../lib/$(LIBZFP) - $(CXX) $(CXXFLAGS) $(INCS) -I../array diffusion.cpp $(CXXLIBS) -o $@ + $(CXX) $(CXXFLAGS) $(INCS) diffusion.cpp $(CXXLIBS) -o $@ $(BINDIR)/diffusionC: diffusionC.o ../lib/$(LIBZFP) ../lib/$(LIBCFP) $(CXX) $(CXXFLAGS) diffusionC.o -lcfp $(CLIBS) -o $@ diffusionC.o: diffusionC.c - $(CC) $(CFLAGS) $(INCS) -I../cfp/include -c diffusionC.c + $(CC) $(CFLAGS) $(INCS) -c diffusionC.c $(BINDIR)/inplace: inplace.c ../lib/$(LIBZFP) $(CC) $(CFLAGS) $(INCS) inplace.c $(CLIBS) -o $@ $(BINDIR)/iterator: iterator.cpp ../lib/$(LIBZFP) - $(CXX) $(CXXFLAGS) $(INCS) -I../array iterator.cpp $(CXXLIBS) -o $@ + $(CXX) $(CXXFLAGS) $(INCS) iterator.cpp $(CXXLIBS) -o $@ + +$(BINDIR)/iteratorC: iteratorC.o ../lib/$(LIBZFP) ../lib/$(LIBCFP) + $(CXX) $(CXXFLAGS) iteratorC.o -lcfp $(CLIBS) -o $@ + +itertorC.o: iteratorC.c + $(CC) $(CFLAGS) $(INCS) -c itertorC.c $(BINDIR)/pgm: pgm.c ../lib/$(LIBZFP) $(CC) $(CFLAGS) $(INCS) pgm.c $(CLIBS) -o $@ @@ -53,4 +60,4 @@ $(BINDIR)/speed: speed.c ../lib/$(LIBZFP) $(CC) $(CFLAGS) $(INCS) speed.c $(CLIBS) -o $@ clean: - rm -f $(TARGETS) diffusionC.o + rm -f $(TARGETS) diffusionC.o iteratorC.o diff --git a/fortran/Makefile b/fortran/Makefile index 229bf42cb..ac55a7ea4 100644 --- a/fortran/Makefile +++ b/fortran/Makefile @@ -2,13 +2,13 @@ include ../Config LIBDIR = ../lib MODDIR = ../modules -TARGETS = $(LIBDIR)/libzFORp.a $(LIBDIR)/libzFORp.so $(MODDIR)/zforp_module.mod -OBJECTS = zfp.o -MODULES = zforp_module.mod +TARGETS = $(LIBDIR)/libzFORp.a $(LIBDIR)/libzFORp.so $(MODDIR)/zfp.mod +OBJECTS = ../src/zfp.o +MODULES = zfp.mod -static: $(LIBDIR)/libzFORp.a $(MODDIR)/zforp_module.mod +static: $(LIBDIR)/libzFORp.a $(MODDIR)/zfp.mod -shared: $(LIBDIR)/libzFORp.so $(MODDIR)/zforp_module.mod +shared: $(LIBDIR)/libzFORp.so $(MODDIR)/zfp.mod clean: rm -f $(TARGETS) $(OBJECTS) @@ -22,9 +22,9 @@ $(LIBDIR)/libzFORp.so: $(OBJECTS) mkdir -p $(LIBDIR) $(FC) $(FFLAGS) -shared $^ -o $@ -$(MODDIR)/zforp_module.mod: $(OBJECTS) +$(MODDIR)/zfp.mod: $(OBJECTS) mkdir -p $(MODDIR) - mv zforp_module.mod $(MODDIR) + mv zfp.mod $(MODDIR) .f.o: $(FC) $(FFLAGS) -c $< diff --git a/include/zfp/cfp.h b/include/zfp/cfp.h index 9d91a1731..c0ba2bbdb 100644 --- a/include/zfp/cfp.h +++ b/include/zfp/cfp.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY -#define CFP_ARRAY +#ifndef CFP_H +#define CFP_H #include #include "zfp/internal/cfp/header.h" diff --git a/include/zfp/codec/generic.hpp b/include/zfp/codec/generic.hpp index 55b398d8d..f66ca38d5 100644 --- a/include/zfp/codec/generic.hpp +++ b/include/zfp/codec/generic.hpp @@ -141,10 +141,10 @@ class generic_base { // unit of allocated data in bytes static size_t alignment() { return sizeof(InternalType); } - static const zfp_type type = zfp::trait::type; // scalar type + static const zfp_type type = zfp::internal::trait::type; // scalar type // zfp::codec::generic_base::header class for array (de)serialization - #include "zfp/internal/array/genheader.hpp" + #include "zfp/internal/codec/genheader.hpp" protected: // pointer to beginning of block diff --git a/include/zfp/codec/zfpcodec.hpp b/include/zfp/codec/zfpcodec.hpp index fd9f4be19..7add3e302 100644 --- a/include/zfp/codec/zfpcodec.hpp +++ b/include/zfp/codec/zfpcodec.hpp @@ -112,7 +112,7 @@ class zfp_base { // unit of allocated data in bytes static size_t alignment() { return stream_alignment() / CHAR_BIT; } - static const zfp_type type = zfp::trait::type; // scalar type + static const zfp_type type = zfp::internal::trait::type; // scalar type // zfp::codec::zfp_base::header class for array (de)serialization #include "zfp/internal/codec/zfpheader.hpp" diff --git a/include/zfp/constarray1.hpp b/include/zfp/constarray1.hpp index 6e339e679..ab5c084c7 100644 --- a/include/zfp/constarray1.hpp +++ b/include/zfp/constarray1.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY1_HPP -#define ZFP_CARRAY1_HPP +#ifndef ZFP_CONSTARRAY1_HPP +#define ZFP_CONSTARRAY1_HPP #include #include diff --git a/include/zfp/constarray2.hpp b/include/zfp/constarray2.hpp index 42335c4d6..a14009b42 100644 --- a/include/zfp/constarray2.hpp +++ b/include/zfp/constarray2.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY2_HPP -#define ZFP_CARRAY2_HPP +#ifndef ZFP_CONSTARRAY2_HPP +#define ZFP_CONSTARRAY2_HPP #include #include diff --git a/include/zfp/constarray3.hpp b/include/zfp/constarray3.hpp index 337420762..09d755fd9 100644 --- a/include/zfp/constarray3.hpp +++ b/include/zfp/constarray3.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY3_HPP -#define ZFP_CARRAY3_HPP +#ifndef ZFP_CONSTARRAY3_HPP +#define ZFP_CONSTARRAY3_HPP #include #include diff --git a/include/zfp/constarray4.hpp b/include/zfp/constarray4.hpp index 9846d1f79..514e3720f 100644 --- a/include/zfp/constarray4.hpp +++ b/include/zfp/constarray4.hpp @@ -1,5 +1,5 @@ -#ifndef ZFP_CARRAY4_HPP -#define ZFP_CARRAY4_HPP +#ifndef ZFP_CONSTARRAY4_HPP +#define ZFP_CONSTARRAY4_HPP #include #include diff --git a/include/zfp/factory.hpp b/include/zfp/factory.hpp index 0b3cce5bb..73091514d 100644 --- a/include/zfp/factory.hpp +++ b/include/zfp/factory.hpp @@ -1,7 +1,7 @@ #ifndef ZFP_FACTORY_HPP #define ZFP_FACTORY_HPP -// ensure zfp/array.h has already been included +// ensure zfp/array.hpp has already been included #ifndef ZFP_ARRAY_HPP #error "zfp/array.hpp must be included before zfp/factory.hpp" #endif diff --git a/include/zfp/internal/array/traits.hpp b/include/zfp/internal/array/traits.hpp index d57ca8c35..7ec4a02b1 100644 --- a/include/zfp/internal/array/traits.hpp +++ b/include/zfp/internal/array/traits.hpp @@ -2,6 +2,7 @@ #define ZFP_TRAITS_HPP namespace zfp { +namespace internal { // useful type traits template @@ -23,6 +24,7 @@ struct trait { static const size_t precision = CHAR_BIT * sizeof(double); }; +} } #endif diff --git a/include/zfp/internal/array/genheader.hpp b/include/zfp/internal/codec/genheader.hpp similarity index 100% rename from include/zfp/internal/array/genheader.hpp rename to include/zfp/internal/codec/genheader.hpp diff --git a/include/zfp/internal/zfp/inline.h b/include/zfp/internal/zfp/inline.h index e9ade3f11..bb10673bb 100644 --- a/include/zfp/internal/zfp/inline.h +++ b/include/zfp/internal/zfp/inline.h @@ -1,5 +1,5 @@ -#ifndef INLINE_H -#define INLINE_H +#ifndef ZFP_INLINE_H +#define ZFP_INLINE_H #ifndef inline_ #if __STDC_VERSION__ >= 199901L diff --git a/tests/Makefile b/tests/Makefile index ef5cfe363..5a80d97a2 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -2,7 +2,7 @@ include ../Config BINDIR = ../bin TARGETS = $(BINDIR)/testzfp $(BINDIR)/testviews -INCS = -I../include -I../array +INCS = -I../include LIBS = -L../lib -lzfp all: $(TARGETS) diff --git a/tests/fortran/testFortran.f b/tests/fortran/testFortran.f index 04b8bb83e..3e1771fca 100644 --- a/tests/fortran/testFortran.f +++ b/tests/fortran/testFortran.f @@ -1,5 +1,5 @@ program main - use zFORp + use zfp use iso_c_binding ! loop counters diff --git a/tests/testzfp.cpp b/tests/testzfp.cpp index 21c26e174..84a21a840 100644 --- a/tests/testzfp.cpp +++ b/tests/testzfp.cpp @@ -759,7 +759,7 @@ test(uint dims, ArraySize array_size) // determine array size uint nx, ny, nz, nw; zfp_field* field = zfp_field_alloc(); - zfp_field_set_type(field, zfp::trait::type); + zfp_field_set_type(field, zfp::internal::trait::type); zfp_field_set_pointer(field, f); switch (dims) { case 1: From 97fa1d75b45578f8f136f9ef64b2a96a00c4382a Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 26 Jul 2022 08:46:53 -0700 Subject: [PATCH 092/277] Disable thread safety checks when OpenMP is absent --- array/zfpcodec.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/array/zfpcodec.h b/array/zfpcodec.h index 098ff36e6..2e00f3342 100644 --- a/array/zfpcodec.h +++ b/array/zfpcodec.h @@ -18,8 +18,10 @@ class zfp_base { protected: // default constructor zfp_base() : - stream(zfp_stream_open(0)), - thread_safety(false) + stream(zfp_stream_open(0)) +#ifdef _OPENMP + , thread_safety(false) +#endif {} // destructor @@ -96,7 +98,11 @@ class zfp_base { bool set_params(uint minbits, uint maxbits, uint maxprec, int maxexp) { return zfp_stream_set_params(stream, minbits, maxbits, maxprec, maxexp) == zfp_true; } // set thread safety mode +#ifdef _OPENMP void set_thread_safety(bool safety) { thread_safety = safety; } +#else + void set_thread_safety(bool) {} +#endif // byte size of codec data structure components indicated by mask size_t size_bytes(uint mask = ZFP_DATA_ALL) const @@ -124,7 +130,9 @@ class zfp_base { stream = zfp_stream_open(0); *stream = *codec.stream; stream->stream = 0; +#ifdef _OPENMP thread_safety = codec.thread_safety; +#endif } // make a thread-local copy of zfp stream and bit stream @@ -182,7 +190,11 @@ class zfp_base { } zfp_stream* stream; // compressed zfp stream +#ifdef _OPENMP bool thread_safety; // thread safety state +#else + static const bool thread_safety = false; // not needed without OpenMP +#endif }; // 1D codec From 81dbdd02dc3d326b0565336c378c526263eba6ac Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 26 Jul 2022 08:47:54 -0700 Subject: [PATCH 093/277] Fix Fortran GNU builds and compiler requirements --- Config | 5 ++++- docs/source/installation.rst | 2 +- docs/source/zforp.rst | 16 +++++++++++----- fortran/Makefile | 16 +++++++++------- 4 files changed, 25 insertions(+), 14 deletions(-) diff --git a/Config b/Config index 08866a1a7..cf0df65df 100644 --- a/Config +++ b/Config @@ -10,7 +10,7 @@ FC = gfortran CSTD = -std=c99 CXXSTD = -std=c++98 # CXXSTD = -std=c++11 - FSTD = -std=f2003 -ffree-form -Wno-c-binding-type + FSTD = -std=f2018 -ffree-form -Wno-c-binding-type # common compiler options ----------------------------------------------------- @@ -24,6 +24,9 @@ SOFLAGS = # do not comment out; use "make ZFP_WITH_OPENMP=0" to disable OpenMP OMPFLAGS = -fopenmp +# Apple clang OpenMP options +# OMPFLAGS = -Xclang -fopenmp + # optional compiler macros ---------------------------------------------------- # use smaller bit stream word type for finer rate granularity; diff --git a/docs/source/installation.rst b/docs/source/installation.rst index d11ed7136..b171f92d2 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -434,4 +434,4 @@ The necessary dependencies can be installed using ``pip`` and the |zfp| Fortran ^^^^^^^ -The optional Fortran bindings require a Fortran 2003 compiler. +The optional Fortran bindings require a Fortran 2018 compiler. diff --git a/docs/source/zforp.rst b/docs/source/zforp.rst index 25ae7613d..532ab8047 100644 --- a/docs/source/zforp.rst +++ b/docs/source/zforp.rst @@ -9,9 +9,11 @@ Fortran Bindings |zfp| |zforprelease| adds |zforp|: a Fortran API providing wrappers around the :ref:`high-level C API `. Wrappers for -:ref:`compressed arrays ` will arrive in a future release. +:ref:`compressed-array classes ` will arrive in a future release. The |zforp| implementation is based on the standard :code:`iso_c_binding` -module available since Fortran 2003. +module available since Fortran 2003. The use of :code:`ptrdiff_t` in +the |zfp| |fieldrelease| C API, however, requires the corresponding +:code:`c_ptrdiff_t` available only since Fortran 2018. Every high-level C API function can be called from a Fortran wrapper function. C structs are wrapped as Fortran derived types, each containing a single C @@ -28,9 +30,13 @@ for how the Fortran API is used to compress and decompress data. .. _zforp_changes: .. note:: - |zfp| |fieldrelease| simplifies the |zforp| module name from zforp_module to zforp. - This will likely require changing associated use statements within existing code when - updating from prior versions of zFORp. + |zfp| |fieldrelease| simplifies the |zforp| module name from + :code:`zforp_module` to :code:`zforp`. This will likely require + changing associated use statements within existing code when updating + from prior versions of zFORp. + + Furthermore, as outlined above, the |zfp| |fieldrelease| API requires + a Fortran 2018 compiler. Types diff --git a/fortran/Makefile b/fortran/Makefile index 229bf42cb..ce1db5d12 100644 --- a/fortran/Makefile +++ b/fortran/Makefile @@ -1,14 +1,16 @@ include ../Config +.SUFFIXES: .f90 + LIBDIR = ../lib MODDIR = ../modules -TARGETS = $(LIBDIR)/libzFORp.a $(LIBDIR)/libzFORp.so $(MODDIR)/zforp_module.mod +TARGETS = $(LIBDIR)/libzFORp.a $(LIBDIR)/libzFORp.so $(MODDIR)/zforp.mod OBJECTS = zfp.o -MODULES = zforp_module.mod +MODULES = zforp.mod -static: $(LIBDIR)/libzFORp.a $(MODDIR)/zforp_module.mod +static: $(LIBDIR)/libzFORp.a $(MODDIR)/zforp.mod -shared: $(LIBDIR)/libzFORp.so $(MODDIR)/zforp_module.mod +shared: $(LIBDIR)/libzFORp.so $(MODDIR)/zforp.mod clean: rm -f $(TARGETS) $(OBJECTS) @@ -22,9 +24,9 @@ $(LIBDIR)/libzFORp.so: $(OBJECTS) mkdir -p $(LIBDIR) $(FC) $(FFLAGS) -shared $^ -o $@ -$(MODDIR)/zforp_module.mod: $(OBJECTS) +$(MODDIR)/zforp.mod: $(OBJECTS) mkdir -p $(MODDIR) - mv zforp_module.mod $(MODDIR) + mv zforp.mod $(MODDIR) -.f.o: +.f90.o: $(FC) $(FFLAGS) -c $< From 54966d1e5a9335eb43195b83faddcb2765ebaead Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Tue, 26 Jul 2022 11:40:44 -0700 Subject: [PATCH 094/277] Fix gmake build issues --- examples/Makefile | 4 ++-- fortran/Makefile | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/examples/Makefile b/examples/Makefile index 588db60f2..17634d7a4 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -44,8 +44,8 @@ $(BINDIR)/iterator: iterator.cpp ../lib/$(LIBZFP) $(BINDIR)/iteratorC: iteratorC.o ../lib/$(LIBZFP) ../lib/$(LIBCFP) $(CXX) $(CXXFLAGS) iteratorC.o -lcfp $(CLIBS) -o $@ -itertorC.o: iteratorC.c - $(CC) $(CFLAGS) $(INCS) -c itertorC.c +iteratorC.o: iteratorC.c + $(CC) $(CFLAGS) $(INCS) -c iteratorC.c $(BINDIR)/pgm: pgm.c ../lib/$(LIBZFP) $(CC) $(CFLAGS) $(INCS) pgm.c $(CLIBS) -o $@ diff --git a/fortran/Makefile b/fortran/Makefile index ce1db5d12..9e5148681 100644 --- a/fortran/Makefile +++ b/fortran/Makefile @@ -4,9 +4,9 @@ include ../Config LIBDIR = ../lib MODDIR = ../modules -TARGETS = $(LIBDIR)/libzFORp.a $(LIBDIR)/libzFORp.so $(MODDIR)/zforp.mod +TARGETS = $(LIBDIR)/libzFORp.a $(LIBDIR)/libzFORp.so $(MODDIR)/zfp.mod OBJECTS = zfp.o -MODULES = zforp.mod +MODULES = zfp.mod static: $(LIBDIR)/libzFORp.a $(MODDIR)/zforp.mod @@ -26,7 +26,7 @@ $(LIBDIR)/libzFORp.so: $(OBJECTS) $(MODDIR)/zforp.mod: $(OBJECTS) mkdir -p $(MODDIR) - mv zforp.mod $(MODDIR) + mv $(MODULES) $(MODDIR) .f90.o: $(FC) $(FFLAGS) -c $< From 0f530708a28ca6a45ce085111691268d5a408656 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 26 Jul 2022 13:04:28 -0700 Subject: [PATCH 095/277] Silence copy constructor warning --- array/zfparray1.h | 1 + array/zfparray2.h | 1 + array/zfparray3.h | 1 + array/zfparray4.h | 1 + 4 files changed, 4 insertions(+) diff --git a/array/zfparray1.h b/array/zfparray1.h index a388fc4cf..9de08aa8c 100644 --- a/array/zfparray1.h +++ b/array/zfparray1.h @@ -79,6 +79,7 @@ class array1 : public array { // copy constructor--performs a deep copy array1(const array1& a) : + array(), cache(store) { deep_copy(a); diff --git a/array/zfparray2.h b/array/zfparray2.h index 7fc285cde..e5fce85b4 100644 --- a/array/zfparray2.h +++ b/array/zfparray2.h @@ -84,6 +84,7 @@ class array2 : public array { // copy constructor--performs a deep copy array2(const array2& a) : + array(), cache(store) { deep_copy(a); diff --git a/array/zfparray3.h b/array/zfparray3.h index 8d5a6a853..83261135c 100644 --- a/array/zfparray3.h +++ b/array/zfparray3.h @@ -86,6 +86,7 @@ class array3 : public array { // copy constructor--performs a deep copy array3(const array3& a) : + array(), cache(store) { deep_copy(a); diff --git a/array/zfparray4.h b/array/zfparray4.h index dbebb9015..9e4b32e42 100644 --- a/array/zfparray4.h +++ b/array/zfparray4.h @@ -88,6 +88,7 @@ class array4 : public array { // copy constructor--performs a deep copy array4(const array4& a) : + array(), cache(store) { deep_copy(a); From 982ca868375cc4aa0ff33f85a25ba60f98af05da Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Tue, 26 Jul 2022 13:56:03 -0700 Subject: [PATCH 096/277] Update cfp header includes, rename main cfp header --- cfp/cfp.cpp | 2 +- cfp/template/cfpheader.cpp | 6 +++--- include/zfp/{cfp.h => array.h} | 4 ++-- include/zfp/internal/cfp/array1d.h | 4 ++-- include/zfp/internal/cfp/array1f.h | 4 ++-- include/zfp/internal/cfp/array2d.h | 4 ++-- include/zfp/internal/cfp/array2f.h | 4 ++-- include/zfp/internal/cfp/array3d.h | 4 ++-- include/zfp/internal/cfp/array3f.h | 4 ++-- include/zfp/internal/cfp/array4d.h | 4 ++-- include/zfp/internal/cfp/array4f.h | 4 ++-- include/zfp/internal/cfp/header.h | 4 ++-- tests/cfp/testCfpArray_source.c | 2 +- tests/cfp/testCfpNamespace.c | 2 +- 14 files changed, 26 insertions(+), 26 deletions(-) rename include/zfp/{cfp.h => array.h} (94%) diff --git a/cfp/cfp.cpp b/cfp/cfp.cpp index 4cbb14d8f..b360760f4 100644 --- a/cfp/cfp.cpp +++ b/cfp/cfp.cpp @@ -1,5 +1,5 @@ #include "cfpheader.cpp" -#include "zfp/cfp.h" +#include "zfp/array.h" #include "cfparray1f.cpp" #include "cfparray1d.cpp" diff --git a/cfp/template/cfpheader.cpp b/cfp/template/cfpheader.cpp index ac6319aa1..b9f619179 100644 --- a/cfp/template/cfpheader.cpp +++ b/cfp/template/cfpheader.cpp @@ -2,6 +2,8 @@ static CFP_HEADER_TYPE _t1(CFP_HEADER_TYPE, ctor_buffer)(const void* data, size_t bytes) { CFP_HEADER_TYPE h; + h.object = 0; + try { // construct generic header and query array type header hdr(data, bytes); @@ -35,9 +37,7 @@ _t1(CFP_HEADER_TYPE, ctor_buffer)(const void* data, size_t bytes) break; } } - catch (...) { - h.object = 0; - } + catch (...) {} return h; } diff --git a/include/zfp/cfp.h b/include/zfp/array.h similarity index 94% rename from include/zfp/cfp.h rename to include/zfp/array.h index c0ba2bbdb..b503abc47 100644 --- a/include/zfp/cfp.h +++ b/include/zfp/array.h @@ -1,5 +1,5 @@ -#ifndef CFP_H -#define CFP_H +#ifndef CFP_ARRAY_H +#define CFP_ARRAY_H #include #include "zfp/internal/cfp/header.h" diff --git a/include/zfp/internal/cfp/array1d.h b/include/zfp/internal/cfp/array1d.h index 155c9d67a..65bddff3b 100644 --- a/include/zfp/internal/cfp/array1d.h +++ b/include/zfp/internal/cfp/array1d.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_1D -#define CFP_ARRAY_1D +#ifndef CFP_ARRAY_1D_H +#define CFP_ARRAY_1D_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array1f.h b/include/zfp/internal/cfp/array1f.h index e271a8c20..3f4d21e4f 100644 --- a/include/zfp/internal/cfp/array1f.h +++ b/include/zfp/internal/cfp/array1f.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_1F -#define CFP_ARRAY_1F +#ifndef CFP_ARRAY_1F_H +#define CFP_ARRAY_1F_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array2d.h b/include/zfp/internal/cfp/array2d.h index 5acbc0c6b..9b070987d 100644 --- a/include/zfp/internal/cfp/array2d.h +++ b/include/zfp/internal/cfp/array2d.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_2D -#define CFP_ARRAY_2D +#ifndef CFP_ARRAY_2D_H +#define CFP_ARRAY_2D_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array2f.h b/include/zfp/internal/cfp/array2f.h index f55a1a656..85bf584e7 100644 --- a/include/zfp/internal/cfp/array2f.h +++ b/include/zfp/internal/cfp/array2f.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_2F -#define CFP_ARRAY_2F +#ifndef CFP_ARRAY_2F_H +#define CFP_ARRAY_2F_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array3d.h b/include/zfp/internal/cfp/array3d.h index 8a4b68fd8..c3c337f6b 100644 --- a/include/zfp/internal/cfp/array3d.h +++ b/include/zfp/internal/cfp/array3d.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_3D -#define CFP_ARRAY_3D +#ifndef CFP_ARRAY_3D_H +#define CFP_ARRAY_3D_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array3f.h b/include/zfp/internal/cfp/array3f.h index b498b7678..43fbe7229 100644 --- a/include/zfp/internal/cfp/array3f.h +++ b/include/zfp/internal/cfp/array3f.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_3F -#define CFP_ARRAY_3F +#ifndef CFP_ARRAY_3F_H +#define CFP_ARRAY_3F_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array4d.h b/include/zfp/internal/cfp/array4d.h index 81bb795ab..22e6a88fd 100644 --- a/include/zfp/internal/cfp/array4d.h +++ b/include/zfp/internal/cfp/array4d.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_4D -#define CFP_ARRAY_4D +#ifndef CFP_ARRAY_4D_H +#define CFP_ARRAY_4D_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/array4f.h b/include/zfp/internal/cfp/array4f.h index b13016153..b5e076748 100644 --- a/include/zfp/internal/cfp/array4f.h +++ b/include/zfp/internal/cfp/array4f.h @@ -1,5 +1,5 @@ -#ifndef CFP_ARRAY_4F -#define CFP_ARRAY_4F +#ifndef CFP_ARRAY_4F_H +#define CFP_ARRAY_4F_H #include #include "zfp.h" diff --git a/include/zfp/internal/cfp/header.h b/include/zfp/internal/cfp/header.h index 7be745bd5..01d78ba54 100644 --- a/include/zfp/internal/cfp/header.h +++ b/include/zfp/internal/cfp/header.h @@ -1,5 +1,5 @@ -#ifndef CFP_HEADER -#define CFP_HEADER +#ifndef CFP_HEADER_H +#define CFP_HEADER_H typedef struct { void* object; diff --git a/tests/cfp/testCfpArray_source.c b/tests/cfp/testCfpArray_source.c index 6731ebbda..de0983412 100644 --- a/tests/cfp/testCfpArray_source.c +++ b/tests/cfp/testCfpArray_source.c @@ -8,7 +8,7 @@ #include #include -#include "zfp/cfp.h" +#include "zfp/array.h" #include "zfp.h" #include "utils/genSmoothRandNums.h" diff --git a/tests/cfp/testCfpNamespace.c b/tests/cfp/testCfpNamespace.c index 3e341cb10..87ac687fd 100644 --- a/tests/cfp/testCfpNamespace.c +++ b/tests/cfp/testCfpNamespace.c @@ -3,7 +3,7 @@ #include #include -#include "zfp/cfp.h" +#include "zfp/array.h" /* only run this test when compiling with CFP_NAMESPACE=cfp2 */ From 725f5cd7f780bce627367e385236c2e598093885 Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Tue, 26 Jul 2022 13:56:33 -0700 Subject: [PATCH 097/277] remove fortran modules directory from git tracking --- .gitignore | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 2f388bbf7..66f13148d 100644 --- a/.gitignore +++ b/.gitignore @@ -6,4 +6,4 @@ lib dist wheelhouse zfpy.egg-info - +modules From 7edcd917d896cfd16eb8c39e14674e84544f3726 Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Tue, 26 Jul 2022 14:04:59 -0700 Subject: [PATCH 098/277] Update time limit on gitlab tests --- tests/gitlab/gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index f0980e0b9..5a7ee9bf5 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -4,7 +4,7 @@ variables: GIT_SUBMODULE_STRATEGY: recursive - LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc" + LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc -t 00:15:00" stages: - build From 73aeb4644e22e3003a2a80b8eb180850a4d1d98b Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Tue, 26 Jul 2022 14:18:10 -0700 Subject: [PATCH 099/277] Fix compiler warnings/errors in examples --- examples/diffusion.cpp | 9 +++++---- examples/diffusionC.c | 2 +- examples/iteratorC.c | 2 +- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/examples/diffusion.cpp b/examples/diffusion.cpp index efe105002..3dae1d611 100644 --- a/examples/diffusion.cpp +++ b/examples/diffusion.cpp @@ -6,6 +6,7 @@ #include #include #include +#include #include "zfp/array2.hpp" #include "zfp/constarray2.hpp" #include "zfp/codec/generic.hpp" @@ -343,7 +344,7 @@ int main(int argc, char* argv[]) config = zfp_config_accuracy(tolerance); } else if (std::string(argv[i]) == "-b") { - if (++i == argc || sscanf(argv[i], "%zu", &cache_size) != 1) + if (++i == argc || (std::istringstream(argv[i]) >> cache_size).fail()) return usage(); cache_size *= 4 * 4 * sizeof(double); } @@ -364,8 +365,8 @@ int main(int argc, char* argv[]) parallel = true; #endif else if (std::string(argv[i]) == "-n") { - if (++i == argc || sscanf(argv[i], "%zu", &nx) != 1 || - ++i == argc || sscanf(argv[i], "%zu", &ny) != 1) + if (++i == argc || (std::istringstream(argv[i]) >> nx).fail() || + ++i == argc || (std::istringstream(argv[i]) >> ny).fail()) return usage(); } else if (std::string(argv[i]) == "-p") { @@ -383,7 +384,7 @@ int main(int argc, char* argv[]) else if (std::string(argv[i]) == "-R") config = zfp_config_reversible(); else if (std::string(argv[i]) == "-t") { - if (++i == argc || sscanf(argv[i], "%zu", &nt) != 1) + if (++i == argc || (std::istringstream(argv[i]) >> nt).fail()) return usage(); } else diff --git a/examples/diffusionC.c b/examples/diffusionC.c index fe90a4d33..e6f9ccb77 100644 --- a/examples/diffusionC.c +++ b/examples/diffusionC.c @@ -7,7 +7,7 @@ forward Euler finite difference solution to the heat equation on a 2D grid #include #include -#include "zfp/cfp.h" +#include "zfp/array.h" #define _ (CFP_NAMESPACE.array2d) #define MAX(x, y) (((nx) > (ny)) ? (nx) : (ny)) diff --git a/examples/iteratorC.c b/examples/iteratorC.c index a2ce139f7..1ca909efc 100644 --- a/examples/iteratorC.c +++ b/examples/iteratorC.c @@ -1,6 +1,6 @@ #include #include -#include "zfp/cfp.h" +#include "zfp/array.h" void print1(cfp_ptr1d p, size_t n) { From c597619cef6810e2f791b071398540d35016c369 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 27 Jul 2022 15:39:27 -0700 Subject: [PATCH 100/277] Fix compiler warnings in tests --- tests/array/decode/testTemplatedDecodeBase.cpp | 10 ++++++---- tests/src/misc/zfpFieldBase.c | 1 - 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/tests/array/decode/testTemplatedDecodeBase.cpp b/tests/array/decode/testTemplatedDecodeBase.cpp index 35873a52b..7e6514dca 100644 --- a/tests/array/decode/testTemplatedDecodeBase.cpp +++ b/tests/array/decode/testTemplatedDecodeBase.cpp @@ -372,9 +372,15 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodeBlock_resultsMatchNonTemplated) TEST(TemplatedDecodeTests, given_TemplatedDecodeBlockStrided_resultsMatchNonTemplated) { size_t countX = 4 * SX; +#if DIMS > 1 size_t countY = SY / SX; +#endif +#if DIMS > 2 size_t countZ = SZ / SY; +#endif +#if DIMS == 4 size_t countW = SW / SZ; +#endif SCALAR* dataArr; populateStridedArray(&dataArr, DUMMY_VAL); @@ -414,7 +420,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodeBlockStrided_resultsMatchNonTemp size_t sz = ZFP_DECODE_BLOCK_STRIDED_FUNC(stream, data1, SX); size_t tsz = decode_block_strided(tstream, data2, SX); - size_t count = countX; #elif DIMS == 2 SCALAR *data1 = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY); ASSERT_TRUE(data1 != nullptr); @@ -424,7 +429,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodeBlockStrided_resultsMatchNonTemp size_t sz = ZFP_DECODE_BLOCK_STRIDED_FUNC(stream, data1, SX, SY); size_t tsz = decode_block_strided(tstream, data2, SX, SY); - size_t count = countX * countY; #elif DIMS == 3 SCALAR *data1 = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ); ASSERT_TRUE(data1 != nullptr); @@ -434,7 +438,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodeBlockStrided_resultsMatchNonTemp size_t sz = ZFP_DECODE_BLOCK_STRIDED_FUNC(stream, data1, SX, SY, SZ); size_t tsz = decode_block_strided(tstream, data2, SX, SY, SZ); - size_t count = countX * countY * countZ; #elif DIMS == 4 SCALAR *data1 = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ * countW); ASSERT_TRUE(data1 != nullptr); @@ -444,7 +447,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodeBlockStrided_resultsMatchNonTemp size_t sz = ZFP_DECODE_BLOCK_STRIDED_FUNC(stream, data1, SX, SY, SZ, SW); size_t tsz = decode_block_strided(tstream, data2, SX, SY, SZ, SW); - size_t count = countX * countY * countZ * countW; #endif ASSERT_TRUE(sz == tsz); diff --git a/tests/src/misc/zfpFieldBase.c b/tests/src/misc/zfpFieldBase.c index fcf2deda2..fa2ddc206 100644 --- a/tests/src/misc/zfpFieldBase.c +++ b/tests/src/misc/zfpFieldBase.c @@ -181,7 +181,6 @@ when_noncontiguousDataWithNegativeStride_fieldBeginsAtCorrectLocation(void **sta { struct setupVars *bundle = *state; zfp_field* field = bundle->field; - SCALAR* data = bundle->data; #if DIMS == 1 ptrdiff_t min = ((int)-SX * (ptrdiff_t)(NX - 1)); From ec96f631f4531bf2202e0af4f3a699d0552c2aca Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 17:45:34 -0700 Subject: [PATCH 101/277] Document lower bound on maxbits in expert mode --- docs/source/modes.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/source/modes.rst b/docs/source/modes.rst index 72d46c802..087fe04ba 100644 --- a/docs/source/modes.rst +++ b/docs/source/modes.rst @@ -99,6 +99,17 @@ terminates as soon as a single constraint is violated (except :c:member:`zfp_stream.minbits`, which is satisfied at the end of encoding by padding zeros). +.. warning:: + + For floating-point data, the :c:member:`zfp_stream.maxbits` parameter must + be large enough to allow the common block exponent and any control bits to + be encoded. This implies *maxbits* |geq| 9 for single-precision data and + *maxbits* |geq| 12 for double-precision data. Choosing a smaller value is + of no use as it would prevent any fraction (value) bits from being encoded, + resulting in an all-zero decompressed block. More importantly, such a + constraint will not be respected by |zfp| for performance reasons, which + if not accounted for could potentially lead to buffer overruns. + As mentioned above, other combinations of constraints can be used. For example, to ensure that the compressed stream is not larger than the uncompressed one, or that it fits within the amount of memory From 8eecbfbb50eb0acdf19448ab475cab37c2347e8d Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 17:52:58 -0700 Subject: [PATCH 102/277] Document new name for zFORp module --- CHANGELOG.md | 7 ++++--- docs/source/zforp.rst | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index eca10cbfc..43fc7d399 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,9 +48,10 @@ few of the API changes, other than to cfp, should impact existing code. - The compressed-array C++ implementation has been completely refactored to make it more modular, extensible, and reusable across array types. - Array block shapes are now computed on the fly rather than stored. -- The cfp API now wraps array objects in structs. -- The zfpy decompression API now supports the more general `memoryview` over - `bytes` objects. +- The cfp C API now wraps array objects in structs. +- The zfpy Python API now supports the more general `memoryview` over + `bytes` objects for decompression. +- The zFORp Fortran module name is now `zfp` instead of `zforp_module`. - Some command-line options for the `diffusion` example have changed. - CMake 3.9 or later is now required for CMake builds. diff --git a/docs/source/zforp.rst b/docs/source/zforp.rst index 532ab8047..5cd0ff4e0 100644 --- a/docs/source/zforp.rst +++ b/docs/source/zforp.rst @@ -31,8 +31,8 @@ for how the Fortran API is used to compress and decompress data. .. note:: |zfp| |fieldrelease| simplifies the |zforp| module name from - :code:`zforp_module` to :code:`zforp`. This will likely require - changing associated use statements within existing code when updating + ``zforp_module`` to ``zfp``. This will likely require changing + associated use statements within existing code when updating from prior versions of zFORp. Furthermore, as outlined above, the |zfp| |fieldrelease| API requires From 1bcbf0ecbe714888ad2434e7a27ce2f73ff91e84 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 17:55:19 -0700 Subject: [PATCH 103/277] Ensure proper make clean in examples --- examples/Makefile | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/examples/Makefile b/examples/Makefile index 17634d7a4..0e288544c 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -16,8 +16,7 @@ CXXLIBS = $(LIBS) $(LDFLAGS) # add cfp examples when BUILD_CFP is enabled ifneq ($(BUILD_CFP),0) - TARGETS += $(BINDIR)/diffusionC\ - $(BINDIR)/iteratorC + TARGETS += $(BINDIR)/diffusionC $(BINDIR)/iteratorC endif @@ -60,4 +59,4 @@ $(BINDIR)/speed: speed.c ../lib/$(LIBZFP) $(CC) $(CFLAGS) $(INCS) speed.c $(CLIBS) -o $@ clean: - rm -f $(TARGETS) diffusionC.o iteratorC.o + rm -f $(TARGETS) $(BINDIR)/diffusionC $(BINDIR)/iteratorC diffusionC.o iteratorC.o From b2ea041601362443f092ccc7b4d0a6443cfeb763 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 17:56:08 -0700 Subject: [PATCH 104/277] Remove -I../array --- tests/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Makefile b/tests/Makefile index fd3b21484..94339dc53 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -2,7 +2,7 @@ include ../Config BINDIR = ../bin TARGETS = $(BINDIR)/testzfp $(BINDIR)/testviews -INCS = -I../include -I../array +INCS = -I../include LIBS = -L../lib -lzfp $(LDFLAGS) all: $(TARGETS) From 1c66f76b707253edb2224a6437c7315cb2b6e514 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 17:56:56 -0700 Subject: [PATCH 105/277] Silence compiler warnings in tests --- tests/array/decode/testTemplatedDecodeBase.cpp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/array/decode/testTemplatedDecodeBase.cpp b/tests/array/decode/testTemplatedDecodeBase.cpp index 7e6514dca..0edb5f04d 100644 --- a/tests/array/decode/testTemplatedDecodeBase.cpp +++ b/tests/array/decode/testTemplatedDecodeBase.cpp @@ -466,9 +466,15 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodeBlockStrided_resultsMatchNonTemp TEST(TemplatedDecodeTests, given_TemplatedDecodePartialBlockStrided_resultsMatchNonTemplated) { size_t countX = 4 * SX; +#if DIMS > 1 size_t countY = SY / SX; +#endif +#if DIMS > 2 size_t countZ = SZ / SY; +#endif +#if DIMS == 4 size_t countW = SW / SZ; +#endif SCALAR* dataArr; populateStridedArray(&dataArr, DUMMY_VAL); From 539db0866954c3cef815b5fd68ab43465cb1472d Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 18:18:29 -0700 Subject: [PATCH 106/277] Update CHANGELOG to reflect directory restructuring --- CHANGELOG.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 43fc7d399..e52bb183f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,12 +25,20 @@ few of the API changes, other than to cfp, should impact existing code. - Additional functions for querying `zfp_field` and `zfp_stream` structs. - `zfp_config`: struct that encapsulates compression mode and parameters. - Rounding modes for reducing bias in compression errors. -- New examples: `ppm` and `iteratorC`. +- New examples: `array`, `iteratorC`, and `ppm`. ### Changed +- Headers from `array/`, `cfp/include`, and `include/` have been renamed + and reorganized into a common `include/` directory. + - The libzfp API is now confined to `zfp.h`, `zfp.hpp`, and `zfp.mod` + for C, C++, and Fortran bindings, respectively. These all appear in + the top-level `include/` directory upon installation. + - C++ headers now use a `.hpp` suffix; C headers use a `.h` suffix. + - C++ headers like `array/zfparray.h` have been renamed `zfp/array.hpp`. + - C headers like `cfp/include/cfparrays.h` have been renamed `zfp/array.h`. - `size_t` and `ptrdiff_t` replace `uint` and `int` for array sizes and - strides in the array classes and C API. + strides in the array classes and C/Fortran APIs. - `zfp_bool` replaces `int` as Boolean type in the C API. - `bitstream_offset` and `bitstream_size` replace `size_t` to ensure support for 64-bit offsets into and lengths of bit streams. Consequently, the From cb618438b35243182de48489b22d06a31e2fa184 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 18:21:52 -0700 Subject: [PATCH 107/277] Correct suffix for examples/array2d.h --- examples/{array2d.h => array2d.hpp} | 4 ++-- examples/diffusion.cpp | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) rename examples/{array2d.h => array2d.hpp} (98%) diff --git a/examples/array2d.h b/examples/array2d.hpp similarity index 98% rename from examples/array2d.h rename to examples/array2d.hpp index 429da5a93..c349328b5 100644 --- a/examples/array2d.h +++ b/examples/array2d.hpp @@ -1,5 +1,5 @@ -#ifndef ARRAY2D_H -#define ARRAY2D_H +#ifndef ARRAY2D_HPP +#define ARRAY2D_HPP #include #include diff --git a/examples/diffusion.cpp b/examples/diffusion.cpp index 3dae1d611..08cf66949 100644 --- a/examples/diffusion.cpp +++ b/examples/diffusion.cpp @@ -10,7 +10,7 @@ #include "zfp/array2.hpp" #include "zfp/constarray2.hpp" #include "zfp/codec/generic.hpp" -#include "array2d.h" +#include "array2d.hpp" // add half precision if compiler supports it #define __STDC_WANT_IEC_60559_TYPES_EXT__ From 91f1823faaf403600f8d74883aaa3e6935005791 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 27 Jul 2022 20:41:04 -0700 Subject: [PATCH 108/277] Update version identifiers for next release --- CHANGELOG.md | 9 +++++---- fortran/zfp.f90 | 8 ++++---- include/zfp/version.h | 6 +++--- src/zfp.c | 2 +- 4 files changed, 13 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e52bb183f..b6367ef36 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,11 +3,12 @@ Change Log --- -## Unreleased +## 1.0.0 (2022-07-XX) -This future release is not ABI compatible with prior releases due to numerous -changes to function signatures and data structures like `zfp_field`. However, -few of the API changes, other than to cfp, should impact existing code. +This release is not ABI compatible with prior releases due to numerous changes +to function signatures and data structures like `zfp_field`. However, few of +the API changes, other than to the cfp C API for compressed arrays, should +impact existing code. ### Added diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index 8b884a5ee..b561abbca 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -51,9 +51,9 @@ module zfp ! constants are hardcoded ! const_xyz holds value, but xyz is the public constant - integer, parameter :: const_zFORp_version_major = 0 - integer, parameter :: const_zFORp_version_minor = 5 - integer, parameter :: const_zFORp_version_patch = 5 + integer, parameter :: const_zFORp_version_major = 1 + integer, parameter :: const_zFORp_version_minor = 0 + integer, parameter :: const_zFORp_version_patch = 0 integer, protected, bind(c, name="zFORp_version_major") :: zFORp_version_major integer, protected, bind(c, name="zFORp_version_minor") :: zFORp_version_minor integer, protected, bind(c, name="zFORp_version_patch") :: zFORp_version_patch @@ -69,7 +69,7 @@ module zfp integer, protected, bind(c, name="zFORp_library_version") :: zFORp_library_version data zFORp_library_version/const_zFORp_library_version/ - character(len = 36), parameter :: zFORp_version_string = 'zfp version 0.5.5 (May 5, 2019)' + character(len = 36), parameter :: zFORp_version_string = 'zfp version 1.0.0 (July XX, 2022)' integer, parameter :: const_zFORp_min_bits = 1 integer, parameter :: const_zFORp_max_bits = 16658 diff --git a/include/zfp/version.h b/include/zfp/version.h index a337ff692..75872a282 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -2,9 +2,9 @@ #define ZFP_VERSION_H /* library version information */ -#define ZFP_VERSION_MAJOR 0 /* library major version number */ -#define ZFP_VERSION_MINOR 5 /* library minor version number */ -#define ZFP_VERSION_PATCH 5 /* library patch version number */ +#define ZFP_VERSION_MAJOR 1 /* library major version number */ +#define ZFP_VERSION_MINOR 0 /* library minor version number */ +#define ZFP_VERSION_PATCH 0 /* library patch version number */ #define ZFP_VERSION_TWEAK 0 /* library tweak version number */ /* defined for work in progress (indicates unofficial release) */ diff --git a/src/zfp.c b/src/zfp.c index 05f507ca1..736c1f857 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -11,7 +11,7 @@ const uint zfp_codec_version = ZFP_CODEC; const uint zfp_library_version = ZFP_VERSION; -const char* const zfp_version_string = "zfp version " ZFP_VERSION_STRING " (May 5, 2019)"; +const char* const zfp_version_string = "zfp version " ZFP_VERSION_STRING " (July XX, 2022)"; /* private functions ------------------------------------------------------- */ From 6825ee2637e495e2701d82ef63de092bd31006fc Mon Sep 17 00:00:00 2001 From: Peter Lindstrom Date: Thu, 28 Jul 2022 10:27:11 -0700 Subject: [PATCH 109/277] Add missing space between literal and identifier --- tests/utils/testMacros.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/utils/testMacros.h b/tests/utils/testMacros.h index 0b2be136a..4791512d0 100644 --- a/tests/utils/testMacros.h +++ b/tests/utils/testMacros.h @@ -23,8 +23,8 @@ // key2: identifies array dimensions // value: checksum // (macro substitutes "printf() && 0" because we want conditional to fail after executing printf) - #define ASSERT_EQ_CHECKSUM(dims, zfpType, computedChecksum, key1, key2) printf("{UINT64C(0x%"PRIx64"), UINT64C(0x%"PRIx64"), UINT64C(0x%"PRIx64")},\n", key1, key2, computedChecksum) - #define COMPARE_NEQ_CHECKSUM(dims, zfpType, computedChecksum, key1, key2) printf("{UINT64C(0x%"PRIx64"), UINT64C(0x%"PRIx64"), UINT64C(0x%"PRIx64")},\n", key1, key2, computedChecksum) && 0 + #define ASSERT_EQ_CHECKSUM(dims, zfpType, computedChecksum, key1, key2) printf("{UINT64C(0x%" PRIx64 "), UINT64C(0x%" PRIx64 "), UINT64C(0x%" PRIx64 ")},\n", key1, key2, computedChecksum) + #define COMPARE_NEQ_CHECKSUM(dims, zfpType, computedChecksum, key1, key2) printf("{UINT64C(0x%" PRIx64 "), UINT64C(0x%" PRIx64 "), UINT64C(0x%" PRIx64 ")},\n", key1, key2, computedChecksum) && 0 #else #define ASSERT_EQ_CHECKSUM(dims, zfpType, computedChecksum, key1, key2) assert_int_equal(computedChecksum, getChecksumByKey(dims, zfpType, key1, key2)) #define COMPARE_NEQ_CHECKSUM(dims, zfpType, computedChecksum, key1, key2) (computedChecksum != getChecksumByKey(dims, zfpType, key1, key2)) From c99a41f4cea66abc953924063510042048a2a4a4 Mon Sep 17 00:00:00 2001 From: Peter Lindstrom Date: Thu, 28 Jul 2022 12:05:45 -0700 Subject: [PATCH 110/277] Fix mismatched memory (de)allocation calls --- include/zfp/index.hpp | 6 +++--- tests/array/encode/testTemplatedEncodeBase.cpp | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/include/zfp/index.hpp b/include/zfp/index.hpp index 6505af218..b84e9b75c 100644 --- a/include/zfp/index.hpp +++ b/include/zfp/index.hpp @@ -72,7 +72,7 @@ class verbatim { } // destructor - ~verbatim() { delete[] data; } + ~verbatim() { zfp::internal::deallocate(data); } // assignment operator--performs a deep copy verbatim& operator=(const verbatim& index) @@ -168,7 +168,7 @@ class hybrid4 { } // destructor - ~hybrid4() { delete[] data; } + ~hybrid4() { zfp::internal::deallocate(data); } // assignment operator--performs a deep copy hybrid4& operator=(const hybrid4& index) @@ -326,7 +326,7 @@ class hybrid8 { } // destructor - ~hybrid8() { delete[] data; } + ~hybrid8() { zfp::internal::deallocate(data); } // assignment operator--performs a deep copy hybrid8& operator=(const hybrid8& index) diff --git a/tests/array/encode/testTemplatedEncodeBase.cpp b/tests/array/encode/testTemplatedEncodeBase.cpp index 08ecd24d0..06538cb7e 100644 --- a/tests/array/encode/testTemplatedEncodeBase.cpp +++ b/tests/array/encode/testTemplatedEncodeBase.cpp @@ -39,7 +39,7 @@ void populateStridedArray(SCALAR** dataArr, SCALAR dummyVal) switch(DIMS) { case 1: countX = BLOCK_SIDE_LEN * SX; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX); + *dataArr = new SCALAR[countX]; ASSERT_TRUE(*dataArr != nullptr); for (i = 0; i < countX; i++) { @@ -58,7 +58,7 @@ void populateStridedArray(SCALAR** dataArr, SCALAR dummyVal) case 2: countX = BLOCK_SIDE_LEN * SX; countY = SY / SX; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY); + *dataArr = new SCALAR[countX * countY]; ASSERT_TRUE(*dataArr != nullptr); for (j = 0; j < countY; j++) { @@ -82,7 +82,7 @@ void populateStridedArray(SCALAR** dataArr, SCALAR dummyVal) countX = BLOCK_SIDE_LEN * SX; countY = SY / SX; countZ = SZ / SY; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ); + *dataArr = new SCALAR[countX * countY * countZ]; ASSERT_TRUE(*dataArr != nullptr); for (k = 0; k < countZ; k++) { @@ -110,7 +110,7 @@ void populateStridedArray(SCALAR** dataArr, SCALAR dummyVal) countY = SY / SX; countZ = SZ / SY; countW = SW / SZ; - *dataArr = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ * countW); + *dataArr = new SCALAR[countX * countY * countZ * countW]; ASSERT_TRUE(*dataArr != nullptr); for (l = 0; l < countW; l++) { From c86be611d0981442659938044ca05cad5171d56b Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 28 Jul 2022 15:09:06 -0700 Subject: [PATCH 111/277] Update zfpy version number --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index b03ff4d66..e760abf31 100644 --- a/setup.py +++ b/setup.py @@ -3,7 +3,7 @@ setup( name="zfpy", - version="0.5.5", + version="1.0.0", author="Peter Lindstrom", author_email="zfp@llnl.gov", url="https://computing.llnl.gov/projects/floating-point-compression", From 962a48bdef6ac1487f360a6d9ecf3886965e6e33 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 29 Jul 2022 10:34:26 -0700 Subject: [PATCH 112/277] update version in license and tests --- LICENSE | 2 +- appveyor.yml | 2 +- include/zfp/version.h | 3 --- 3 files changed, 2 insertions(+), 5 deletions(-) diff --git a/LICENSE b/LICENSE index f371f8825..9bed13fb4 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ BSD 3-Clause License -Copyright (c) 2014-2019, Lawrence Livermore National Security, LLC +Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/appveyor.yml b/appveyor.yml index 18963b02c..abd9385af 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -1,4 +1,4 @@ -version: 0.5.5-{build} +version: 1.0.0-{build} environment: # zfpy only build for Release builds (otherwise need debug python libs python27_d.lib) diff --git a/include/zfp/version.h b/include/zfp/version.h index 75872a282..790927f30 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -7,9 +7,6 @@ #define ZFP_VERSION_PATCH 0 /* library patch version number */ #define ZFP_VERSION_TWEAK 0 /* library tweak version number */ -/* defined for work in progress (indicates unofficial release) */ -#define ZFP_VERSION_DEVELOP 1 - /* codec version number (see also zfp_codec_version) */ #define ZFP_CODEC 5 From 9a9a226374396454302ff765ba7b58d3faf45b5a Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 29 Jul 2022 10:35:17 -0700 Subject: [PATCH 113/277] update doc versions and add some missing changes --- docs/source/conf.py | 6 +- docs/source/defs.rst | 18 +++--- docs/source/high-level-api.rst | 26 ++++----- docs/source/installation.rst | 15 ++--- docs/source/license.rst | 2 +- docs/source/versions.rst | 101 +++++++++++++++++++++++++++++++++ 6 files changed, 128 insertions(+), 40 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index e44cef318..f6a847a03 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -46,7 +46,7 @@ # General information about the project. project = u'zfp' -copyright = u'2014-2019, LLNL-CODE-663824' +copyright = u'2014-2022, LLNL-CODE-663824' author = u'Peter Lindstrom' # The version info for the project you're documenting, acts as replacement for @@ -54,9 +54,9 @@ # built documents. # # The short X.Y version. -version = u'0.5' +version = u'1.0' # The full version, including alpha/beta/rc tags. -release = u'0.5.5' +release = u'1.0.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/source/defs.rst b/docs/source/defs.rst index ada0f728d..0954f13f4 100644 --- a/docs/source/defs.rst +++ b/docs/source/defs.rst @@ -30,12 +30,12 @@ .. |zforprelease| replace:: 0.5.5 .. |zfpyrelease| replace:: 0.5.5 .. |csizerelease| replace:: 0.5.5 -.. |crpirelease| replace:: 0.5.6 -.. |raiterrelease| replace:: 0.5.6 -.. |64bitrelease| replace:: 0.5.6 -.. |boolrelease| replace:: 0.5.6 -.. |4darrrelease| replace:: 0.5.6 -.. |fieldrelease| replace:: 0.5.6 -.. |carrrelease| replace:: 0.5.6 -.. |cpprelease| replace:: 0.5.6 -.. |verrelease| replace:: 0.5.6 +.. |crpirelease| replace:: 1.0.0 +.. |raiterrelease| replace:: 1.0.0 +.. |64bitrelease| replace:: 1.0.0 +.. |boolrelease| replace:: 1.0.0 +.. |4darrrelease| replace:: 1.0.0 +.. |fieldrelease| replace:: 1.0.0 +.. |carrrelease| replace:: 1.0.0 +.. |cpprelease| replace:: 1.0.0 +.. |verrelease| replace:: 1.0.0 diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 263310757..165beb872 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -58,7 +58,7 @@ Macros :c:macro:`ZFP_VERSION_STRING`, respectively. Available as of |zfp| |64bitrelease|, these macros may be used by applications to test for a certain |zfp| version number, e.g., - :code:`#if ZFP_VERSION >= ZFP_MAKE_VERSION(0, 5, 6)`. + :code:`#if ZFP_VERSION >= ZFP_MAKE_VERSION(1, 0, 0)`. ---- @@ -70,7 +70,7 @@ Macros used by intermediate develop versions. Available as of |zfp| |verrelease|, these macros may be used by applications to test for a certain |zfp| version number, e.g., - :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(0, 5, 6, 1)`. + :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(1, 0, 0, 1)`. ---- @@ -236,14 +236,20 @@ Types The :c:type:`zfp_stream` also stores information about how to execute compression, e.g., sequentially or in parallel. The execution is determined by the policy and any policy-specific parameters such as number of - threads. + threads. :: typedef struct { zfp_exec_policy policy; // execution policy (serial, omp, ...) - zfp_exec_params params; // execution parameters + void* params; // execution parameters } zfp_execution; +.. warning:: + As of |zfp| |verrelease| `zfp_execution` replaces `zfp_exec_params` with a + ``void *`` to the associated `zfp_exec_params` type (e.g. + `zfp_exec_params_omp`) to limit ABI-breaking changes due to future + extensions to |zfp| execution policies. + ---- .. c:type:: zfp_exec_policy @@ -260,18 +266,6 @@ Types ---- -.. c:type:: zfp_exec_params - - Execution parameters are shared among policies in a union. Currently - the only parameters available are for OpenMP. - :: - - typedef union { - zfp_exec_params_omp omp; // OpenMP parameters - } zfp_exec_params; - ----- - .. c:type:: zfp_exec_params_omp Execution parameters for OpenMP parallel compression. These are diff --git a/docs/source/installation.rst b/docs/source/installation.rst index b171f92d2..3aec09ceb 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -15,13 +15,6 @@ they call the compression library, applications must link with |libzfp|. |zfp| is preferably built using `CMake `__, although the core library can also be built using GNU make on Linux, macOS, and MinGW. -|zfp| has successfully been built and tested using these compilers: - -* gcc versions 4.4.7, 4.7.3, 4.8.5, 4.9.4, 5.5.0, 6.1.0, 6.4.0, 7.1.0, 7.3.0, 8.1.0 -* icc versions 14.0.3, 15.0.6, 16.0.4, 17.0.2, 18.0.2, 19.0.3 -* clang versions 3.9.1, 4.0.0, 5.0.0, 6.0.0 -* MinGW version 5.3.0 -* Visual Studio versions 14 (2015), 15 (2017) |zfp| conforms to various language standards, including C89, C99, C++98, C++11, and C++14. @@ -37,7 +30,7 @@ CMake Builds To build |zfp| using `CMake `__ on Linux or macOS, start a Unix shell and type:: - cd zfp-0.5.5 + cd zfp-1.0.0 mkdir build cd build cmake .. @@ -55,7 +48,7 @@ By default, CMake builds will attempt to locate and use To build |zfp| using Visual Studio on Windows, start a DOS shell and type:: - cd zfp-0.5.5 + cd zfp-1.0.0 mkdir build cd build cmake .. @@ -74,7 +67,7 @@ GNU Builds To build |zfp| using `gcc `__ without `OpenMP `__, type:: - cd zfp-0.5.5 + cd zfp-1.0.0 gmake This builds |libzfp| as a static library as well as the |zfp| @@ -161,7 +154,7 @@ Regardless of the settings below, |libzfp| will always be built. .. c:macro:: BUILD_ZFORP Build |libzforp| for Fortran bindings to the C API. Requires Fortran - standard 2003 or later. GNU make users may specify the Fortran compiler + standard 2018 or later. GNU make users may specify the Fortran compiler to use via :: diff --git a/docs/source/license.rst b/docs/source/license.rst index 6ac2a7a93..4d7cff37c 100644 --- a/docs/source/license.rst +++ b/docs/source/license.rst @@ -3,7 +3,7 @@ License ======= -| Copyright (c) 2014-2019, Lawrence Livermore National Security, LLC. +| Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC. | Produced at the Lawrence Livermore National Laboratory. | Written by Peter Lindstrom, Markus Salasoo, Matt Larsen, and Stephen Herbein. | LLNL-CODE-663824. diff --git a/docs/source/versions.rst b/docs/source/versions.rst index 389fd5210..747ad5830 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -3,6 +3,107 @@ Release Notes ============= +zfp 1.0.0, July XX, 2022 + +This release is not ABI compatible with prior releases due to numerous changes +to function signatures and data structures like zfp_fieldu. However, few of +the API changes, other than to the cfp C API for compressed arrays, should +impact existing code. + + +- Added read-only variable-rate array that supports fixed-precision, + fixed-accuracy, and reversible modes. + +- Added compressed-array classes for 4D data. + +- Added const versions of array references, pointers, and iterators. + +- Added a more complete API for pointers and iterators. + +- Added support for pointers and iterators into array views. + +- zfp::array::size_bytes() allows querying the size of different components + of an array object (e.g., payload, cache, index, metadata, ...). + +- Added templated C++ wrappers around the low-level C API. + +- Added a generic codec for storing blocks of uncompressed scalars in zfp's + C++ arrays. + +- Added additional functions for querying zfp_field and zfp_stream structs. + +- Added zfp_config struct that encapsulates compression mode and parameters. + +- Added rounding modes for reducing bias in compression errors. + +- Added new examples: array, iteratorC, and ppm. + + - Changes: + + - Headers from array/, cfp/include, and include/ have been renamed + and reorganized into a common include/ directory. + + - The libzfp API is now confined to zfp.h, zfp.hpp, and zfp.mod + for C, C++, and Fortran bindings, respectively. These all appear in + the top-level include/ directory upon installation. + - C++ headers now use a .hpp suffix; C headers use a .h suffix. + - C++ headers like array/zfparray.h have been renamed zfp/array.hpp. + - C headers like cfp/include/cfparrays.h have been renamed zfp/array.h. + + - size_t and ptrdiff_t replace uint and int for array sizes and + strides in the array classes and C/Fortran APIs. + - zfp_bool replaces int as Boolean type in the C API. + - bitstream_offset and bitstream_size replace size_t to ensure support + for 64-bit offsets into and lengths of bit streams. Consequently, the + bitstream API has changed accordingly. + - All array and view iterators are now random-access iterators. + - Array inspectors now return const_reference rather than a scalar + type like float to allow obtaining a const_pointer to an element + of an immutable array. + - zfp::array::compressed_data() now returns void* instead of uchar*. + - The array (de)serialization API has been revised, resulting in new + zfp::array::header and zfp::exception classes with new exception + messages. + - The array codec class is now responsible for all details regarding + compression. + - The compressed-array C++ implementation has been completely refactored to + make it more modular, extensible, and reusable across array types. + - Array block shapes are now computed on the fly rather than stored. + - The cfp C API now wraps array objects in structs. + - The zfpy Python API now supports the more general memoryview over + bytes objects for decompression. + - The zFORp Fortran module name is now zfp instead of zforp_module. + - Some command-line options for the diffusion example have changed. + - CMake 3.9 or later is now required for CMake builds. + + - Removed: + + - zfp::array::get_header() has been replaced with a zfp::array::header + constructor that accepts an array object. + - ZFP_VERSION_RELEASE is no longer defined (use ZFP_VERSION_PATCH). + + - Bug fixes: + + - make install overwrites googletest. + - Incorrect order of parameters in CUDA memset(). + - C++ compiler warns when __STDC_VERSION__ is undefined. + - CXXFLAGS is misspelled in cfp/src/Makefile. + - zfp_stream_maximum_size() underestimates size in reversible mode. + - Incorrect private_view reads due to missing writeback. + - Unused CPython array is incompatible with PyPy. + - PGI compiler bug causes issues with memory alignment. + - All-subnormal blocks may cause floating-point overflow. + - CUDA bit offsets are limited to 32 bits. + - make install does not install zfp command-line utility. + - OpenMP bit offsets are limited to 32 bits. + - make install does not install Fortran module. + - Reversible mode reports incorrect compressed block size. + - cmocka tests do not build on macOS. + - Thread safety is broken in private_view and private_const_view. + - ZFP_MAX_BITS is off by one. + - diffusionC, iteratorC are not being built with gmake. + + zfp 0.5.5, May 5, 2019 - Added support for reversible (lossless) compression of floating-point and From 44d4f849d60358043bdeb024987609d0823040dd Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 15:34:38 -0700 Subject: [PATCH 114/277] Remove unused variable in decoder test --- tests/array/decode/testTemplatedDecodeBase.cpp | 4 ---- 1 file changed, 4 deletions(-) diff --git a/tests/array/decode/testTemplatedDecodeBase.cpp b/tests/array/decode/testTemplatedDecodeBase.cpp index 0edb5f04d..471c75d35 100644 --- a/tests/array/decode/testTemplatedDecodeBase.cpp +++ b/tests/array/decode/testTemplatedDecodeBase.cpp @@ -514,7 +514,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodePartialBlockStrided_resultsMatch size_t d_sz = ZFP_DECODE_PARTIAL_BLOCK_STRIDED_FUNC(stream, data1, PX, SX); size_t d_tsz = decode_partial_block_strided(tstream, data2, PX, SX); - size_t count = countX; #elif DIMS == 2 SCALAR *data1 = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY); ASSERT_TRUE(data1 != nullptr); @@ -524,7 +523,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodePartialBlockStrided_resultsMatch size_t d_sz = ZFP_DECODE_PARTIAL_BLOCK_STRIDED_FUNC(stream, data1, PX, PY, SX, SY); size_t d_tsz = decode_partial_block_strided(tstream, data2, PX, PY, SX, SY); - size_t count = countX * countY; #elif DIMS == 3 SCALAR *data1 = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ); ASSERT_TRUE(data1 != nullptr); @@ -534,7 +532,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodePartialBlockStrided_resultsMatch size_t d_sz = ZFP_DECODE_PARTIAL_BLOCK_STRIDED_FUNC(stream, data1, PX, PY, PZ, SX, SY, SZ); size_t d_tsz = decode_partial_block_strided(tstream, data2, PX, PY, PZ, SX, SY, SZ); - size_t count = countX * countY * countZ; #elif DIMS == 4 SCALAR *data1 = (SCALAR*)malloc(sizeof(SCALAR) * countX * countY * countZ * countW); ASSERT_TRUE(data1 != nullptr); @@ -544,7 +541,6 @@ TEST(TemplatedDecodeTests, given_TemplatedDecodePartialBlockStrided_resultsMatch size_t d_sz = ZFP_DECODE_PARTIAL_BLOCK_STRIDED_FUNC(stream, data1, PX, PY, PZ, PW, SX, SY, SZ, SW); size_t d_tsz = decode_partial_block_strided(tstream, data2, PX, PY, PZ, PW, SX, SY, SZ, SW); - size_t count = countX * countY * countZ * countW; #endif ASSERT_TRUE(d_sz == d_tsz); From 4ce851be7edebe208d931b66f890a13f20e72dfc Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 17:17:15 -0700 Subject: [PATCH 115/277] Update versions.rst to align with CHANGELOG.md --- CHANGELOG.md | 9 +- docs/source/versions.rst | 654 ++++++++++++++++++++------------------- 2 files changed, 344 insertions(+), 319 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b6367ef36..0e4f9be92 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,8 @@ Change Log This release is not ABI compatible with prior releases due to numerous changes to function signatures and data structures like `zfp_field`. However, few of the API changes, other than to the cfp C API for compressed arrays, should -impact existing code. +impact existing code. Note that numerous header files have been renamed or +moved relative to prior versions. ### Added @@ -30,7 +31,7 @@ impact existing code. ### Changed -- Headers from `array/`, `cfp/include`, and `include/` have been renamed +- Headers from `array/`, `cfp/include/`, and `include/` have been renamed and reorganized into a common `include/` directory. - The libzfp API is now confined to `zfp.h`, `zfp.hpp`, and `zfp.mod` for C, C++, and Fortran bindings, respectively. These all appear in @@ -134,7 +135,7 @@ impact existing code. - Execution policy now applies to both compression and decompression. - Compressed array accessors now return Scalar type instead of - const Scalar& to avoid stale references to evicted cache lines. + `const Scalar&` to avoid stale references to evicted cache lines. ### Fixed @@ -320,7 +321,7 @@ not backward compatible with previous versions of zfp. ### Fixed - Rare bug caused by exponent underflow in blocks with no normal and some - denormal numbers. + subnormal numbers. --- diff --git a/docs/source/versions.rst b/docs/source/versions.rst index 747ad5830..018266bdf 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -3,404 +3,428 @@ Release Notes ============= -zfp 1.0.0, July XX, 2022 +1.0.0 (2022-07-XX) +------------------ This release is not ABI compatible with prior releases due to numerous changes to function signatures and data structures like zfp_fieldu. However, few of -the API changes, other than to the cfp C API for compressed arrays, should -impact existing code. - - -- Added read-only variable-rate array that supports fixed-precision, - fixed-accuracy, and reversible modes. - -- Added compressed-array classes for 4D data. - -- Added const versions of array references, pointers, and iterators. - -- Added a more complete API for pointers and iterators. - -- Added support for pointers and iterators into array views. - -- zfp::array::size_bytes() allows querying the size of different components +the API changes, other than to the |cfp| C API for compressed arrays, should +impact existing code. Note that numerous header files have been renamed or +moved relative to prior versions. + +**Added** + +- ``zfp::const_array``: read-only variable-rate array that supports + fixed-precision, fixed-accuracy, and reversible modes. +- Compressed-array classes for 4D data. +- ``const`` versions of array references, pointers, and iterators. +- A more complete API for pointers and iterators. +- Support for pointers and iterators into array views. +- ``zfp::array::size_bytes()`` allows querying the size of different components of an array object (e.g., payload, cache, index, metadata, ...). - -- Added templated C++ wrappers around the low-level C API. - -- Added a generic codec for storing blocks of uncompressed scalars in zfp's +- Templated C++ wrappers around the low-level C API. +- A generic codec for storing blocks of uncompressed scalars in |zfp|'s C++ arrays. +- Additional functions for querying ``zfp_field`` and ``zfp_stream`` structs. +- ``zfp_config``: struct that encapsulates compression mode and parameters. +- Rounding modes for reducing bias in compression errors. +- New examples: ``array``, ``iteratorC``, and ``ppm``. + +**Changed** + +- Headers from ``array/``, ``cfp/include/``, and ``include/`` have been renamed + and reorganized into a common ``include/`` directory. + + * The libzfp API is now confined to ``zfp.h``, ``zfp.hpp``, and ``zfp.mod`` + for C, C++, and Fortran bindings, respectively. These all appear in + the top-level ``include/`` directory upon installation. + * C++ headers now use a ``.hpp`` suffix; C headers use a ``.h`` suffix. + * C++ headers like ``array/zfparray.h`` have been renamed ``zfp/array.hpp``. + * C headers like ``cfp/include/cfparrays.h`` have been renamed + ``zfp/array.h``. + +- ``size_t`` and ``ptrdiff_t`` replace ``uint`` and ``int`` for array sizes and + strides in the array classes and C/Fortran APIs. +- ``zfp_bool`` replaces ``int`` as Boolean type in the C API. +- ``bitstream_offset`` and ``bitstream_size`` replace ``size_t`` to ensure + support for 64-bit offsets into and lengths of bit streams. Consequently, + the ``bitstream`` API has changed accordingly. +- All array and view iterators are now random-access iterators. +- Array inspectors now return ``const_reference`` rather than a scalar + type like ``float`` to allow obtaining a ``const_pointer`` to an element + of an immutable array. +- ``zfp::array::compressed_data()`` now returns ``void*`` instead of + ``uchar*``. +- The array (de)serialization API has been revised, resulting in new + ``zfp::array::header`` and ``zfp::exception`` classes with new exception + messages. +- The array ``codec`` class is now responsible for all details regarding + compression. +- The compressed-array C++ implementation has been completely refactored to + make it more modular, extensible, and reusable across array types. +- Array block shapes are now computed on the fly rather than stored. +- The |cfp| C API now wraps array objects in structs. +- The |zfpy| Python API now supports the more general ``memoryview`` over + ``bytes`` objects for decompression. +- The zFORp Fortran module name is now ``zfp`` instead of ``zforp_module``. +- Some command-line options for the ``diffusion`` example have changed. +- CMake 3.9 or later is now required for CMake builds. + +**Removed** + +- ``zfp::array::get_header()`` has been replaced with a ``zfp::array::header`` + constructor that accepts an array object. +- ``ZFP_VERSION_RELEASE`` is no longer defined (use ``ZFP_VERSION_PATCH``). + +**Fixed** + +- #66: ``make install`` overwrites googletest. +- #84: Incorrect order of parameters in CUDA ``memset()``. +- #86: C++ compiler warns when ``__STDC_VERSION__`` is undefined. +- #87: ``CXXFLAGS`` is misspelled in ``cfp/src/Makefile``. +- #98: ``zfp_stream_maximum_size()`` underestimates size in reversible mode. +- #99: Incorrect ``private_view`` reads due to missing writeback. +- #109: Unused CPython array is incompatible with PyPy. +- #112: PGI compiler bug causes issues with memory alignment. +- #119: All-subnormal blocks may cause floating-point overflow. +- #121: CUDA bit offsets are limited to 32 bits. +- #122: ``make install`` does not install |zfp| command-line utility. +- #125: OpenMP bit offsets are limited to 32 bits. +- #126: ``make install`` does not install Fortran module. +- #127: Reversible mode reports incorrect compressed block size. +- #150: cmocka tests do not build on macOS. +- #154: Thread safety is broken in ``private_view`` and ``private_const_view``. +- ``ZFP_MAX_BITS`` is off by one. +- ``diffusionC``, ``iteratorC`` are not being built with ``gmake``. + +---- + +0.5.5 (2019-05-05) +------------------ + +**Added** + +- Support for reversible (lossless) compression of floating-point and + integer data. +- Methods for serializing and deserializing |zfp|'s compressed arrays. +- Python bindings for compressing NumPy arrays. +- Fortran bindings to |zfp|'s high-level C API. + +**Changed** + +- The default compressed-array cache size is now a function of the total + number of array elements, irrespective of array shape. + +**Fixed** + +- Incorrect handling of execution policy in |zfp| utility. +- Incorrect handling of decompression via header in |zfp| utility. +- Incorrect cleanup of device memory in CUDA decompress. +- Missing tests for failing mallocs. +- CMake does not install CFP when built. +- ``zfp_write_header()`` and ``zfp_field_metadata()`` succeed even if array + dimensions are too large to fit in header. + +---- + +0.5.4 (2018-10-01) +------------------ + +**Added** + +- Support for CUDA fixed-rate compression and decompression. +- Views into compressed arrays for thread safety, nested array indexing, + slicing, and array subsetting. +- C language bindings for compressed arrays. +- Support for compressing and decompressing 4D data. + +**Changed** + +- Execution policy now applies to both compression and decompression. +- Compressed array accessors now return Scalar type instead of + ``const Scalar&`` to avoid stale references to evicted cache lines. + +**Fixed** + +- Incorrect handling of negative strides. +- Incorrect handling of arrays with more than 2\ :sup:`32` elements in |zfp| + command-line tool. +- ``bitstream`` is not C++ compatible. +- Minimum cache size request is not respected. + +---- + +0.5.3 (2018-03-28) +------------------ -- Added additional functions for querying zfp_field and zfp_stream structs. - -- Added zfp_config struct that encapsulates compression mode and parameters. - -- Added rounding modes for reducing bias in compression errors. - -- Added new examples: array, iteratorC, and ppm. - - - Changes: - - - Headers from array/, cfp/include, and include/ have been renamed - and reorganized into a common include/ directory. - - - The libzfp API is now confined to zfp.h, zfp.hpp, and zfp.mod - for C, C++, and Fortran bindings, respectively. These all appear in - the top-level include/ directory upon installation. - - C++ headers now use a .hpp suffix; C headers use a .h suffix. - - C++ headers like array/zfparray.h have been renamed zfp/array.hpp. - - C headers like cfp/include/cfparrays.h have been renamed zfp/array.h. - - - size_t and ptrdiff_t replace uint and int for array sizes and - strides in the array classes and C/Fortran APIs. - - zfp_bool replaces int as Boolean type in the C API. - - bitstream_offset and bitstream_size replace size_t to ensure support - for 64-bit offsets into and lengths of bit streams. Consequently, the - bitstream API has changed accordingly. - - All array and view iterators are now random-access iterators. - - Array inspectors now return const_reference rather than a scalar - type like float to allow obtaining a const_pointer to an element - of an immutable array. - - zfp::array::compressed_data() now returns void* instead of uchar*. - - The array (de)serialization API has been revised, resulting in new - zfp::array::header and zfp::exception classes with new exception - messages. - - The array codec class is now responsible for all details regarding - compression. - - The compressed-array C++ implementation has been completely refactored to - make it more modular, extensible, and reusable across array types. - - Array block shapes are now computed on the fly rather than stored. - - The cfp C API now wraps array objects in structs. - - The zfpy Python API now supports the more general memoryview over - bytes objects for decompression. - - The zFORp Fortran module name is now zfp instead of zforp_module. - - Some command-line options for the diffusion example have changed. - - CMake 3.9 or later is now required for CMake builds. - - - Removed: - - - zfp::array::get_header() has been replaced with a zfp::array::header - constructor that accepts an array object. - - ZFP_VERSION_RELEASE is no longer defined (use ZFP_VERSION_PATCH). - - - Bug fixes: - - - make install overwrites googletest. - - Incorrect order of parameters in CUDA memset(). - - C++ compiler warns when __STDC_VERSION__ is undefined. - - CXXFLAGS is misspelled in cfp/src/Makefile. - - zfp_stream_maximum_size() underestimates size in reversible mode. - - Incorrect private_view reads due to missing writeback. - - Unused CPython array is incompatible with PyPy. - - PGI compiler bug causes issues with memory alignment. - - All-subnormal blocks may cause floating-point overflow. - - CUDA bit offsets are limited to 32 bits. - - make install does not install zfp command-line utility. - - OpenMP bit offsets are limited to 32 bits. - - make install does not install Fortran module. - - Reversible mode reports incorrect compressed block size. - - cmocka tests do not build on macOS. - - Thread safety is broken in private_view and private_const_view. - - ZFP_MAX_BITS is off by one. - - diffusionC, iteratorC are not being built with gmake. - - -zfp 0.5.5, May 5, 2019 - - - Added support for reversible (lossless) compression of floating-point and - integer data. - - - Added methods for serializing and deserializing zfp's compressed arrays. - - - Added Python bindings for compressing NumPy arrays. - - - Added Fortran bindings to zfp's high-level C API. - - - Change: - - - The default compressed-array cache size is now a function of the total - number of array elements, irrespective of array shape. - - - Bug fixes: - - - Incorrect handling of execution policy in zfp utility. - - Incorrect handling of decompression via header in zfp utility. - - Incorrect cleanup of device memory in CUDA decompress. - - Tests for failing mallocs. - - CMake installation of CFP when built. - - zfp_write_header and zfp_field_metadata now fail if array dimensions - are too large to fit in header. - - -zfp 0.5.4, October 1, 2018 - - - Added support for CUDA fixed-rate compression and decompression. - - - Added views into compressed arrays for thread safety, nested array - indexing, slicing, and array subsetting. - - - Added C language bindings for compressed arrays. - - - Added support for compressing and decompressing 4D data. - - - Changes: - - - Execution policy now applies to both compression and decompression. - - Compressed array accessors now return Scalar type instead of - const Scalar& to avoid stale references to evicted cache lines. - - - Bug fixes: - - - Handling of negative strides. - - Command line tool handling of arrays with more than 2\ :sup:`32` elements. - - bitstream C++ compatibility. - - Respect minimum cache size request. - - -zfp 0.5.3, March 28, 2018 - - - Added support for OpenMP multithreaded compression (but not decompression). - - - Added options for OpenMP execution to zfp command-line tool. - - - Changed return value of zfp_decompress to indicate the number of compressed - bytes processed so far (now returns same value as zfp_compress on success). - - - Added compressed array support for copy construction and assignment via - deep copies. - - - Added virtual destructors to enable inheritance from zfp arrays. - - -zfp 0.5.2, September 28, 2017 - - - Added iterators and proxy objects for pointers and references. - - - Added example illustrating how to use iterators and pointers. - - - Modified diffusion example to optionally use iterators. - - - Moved internal headers under array to array/zfp. - - - Modified 64-bit integer typedefs to avoid the C89 non-compliant long long - and allow for user-supplied types and literal suffixes. - - - Renamed compile-time macros that did not have a ZFP prefix. - - - Fixed issue with setting stream word type via CMake. - - - Rewrote documentation in reStructuredText and added complete - documentation of all public functions, classes, types, and macros. - Removed ASCII documentation. - +**Added** -zfp 0.5.1, March 28, 2017 +- Support for OpenMP multithreaded compression (but not decompression). +- Options for OpenMP execution in |zfp| command-line tool. +- Compressed-array support for copy construction and assignment via deep + copies. +- Virtual destructors to enable inheritance from |zfp| arrays. - - This release primarily fixes a few minor issues but also includes - changes in anticipation of a large number of planned future additions - to the library. No changes have been made to the compressed format, - which is backwards compatible with version 0.5.0. +**Changed** - - Added high-level API support for integer types. +- ``zfp_decompress()`` now returns the number of compressed bytes processed so + far, i.e., the same value returned by ``zfp_compress()``. - - Separated library version from CODEC version and added version string. +---- - - Added example that illustrates in-place compression. +0.5.2 (2017-09-28) +------------------ - - Added support for CMake builds. +**Added** - - Corrected inconsistent naming of BIT_STREAM macros in code and - documentation. +- Iterators and proxy objects for pointers and references. +- Example illustrating how to use iterators and pointers. - - Renamed some of the header bit mask macros. +**Changed** - - Added return values to stream_skip and stream_flush to indicate the - number of bits skipped or output. +- Diffusion example now optionally uses iterators. +- Moved internal headers under array to ``array/zfp``. +- Modified 64-bit integer typedefs to avoid the C89 non-compliant ``long long`` + and allow for user-supplied types and literal suffixes. +- Renamed compile-time macros that did not have a ``ZFP`` prefix. +- Rewrote documentation in reStructuredText and added complete documentation + of all public functions, classes, types, and macros. - - Renamed stream_block and stream_delta to make it clear that they refer - to strided streams. Added missing definition of stream_stride_block. +**Fixed** - - Changed int/uint types in places to use ptrdiff_t/size_t where - appropriate. +- Issue with setting stream word type via CMake. - - Changed API for zfp_set_precision and zfp_set_accuracy to not require - the scalar type. +---- - - Added missing static keyword in decode_block. +0.5.1 (2017-03-28) +------------------ - - Changed testzfp to allow specifying which tests to perform on the - command line. +This release primarily fixes a few minor issues but also includes changes in +anticipation of a large number of planned future additions to the library. +No changes have been made to the compressed format, which is backwards +compatible with version 0.5.0. - - Fixed bug that prevented defining uninitialized arrays. +**Added** - - Fixed incorrect computation of array sizes in zfp_field_size. +- High-level API support for integer types. +- Example that illustrates in-place compression. +- Support for CMake builds. +- Documentation that discusses common issues with using |zfp|. - - Fixed minor issues that prevented code from compiling on Windows. +**Changed** - - Fixed issue with fixed-accuracy headers that caused unnecessary storage. +- Separated library version from CODEC version and added version string. +- Corrected inconsistent naming of ``BIT_STREAM`` macros in code and + documentation. +- Renamed some of the header bit mask macros. +- ``stream_skip()`` and ``stream_flush()`` now return the number of bits + skipped or output. +- Renamed ``stream_block()`` and ``stream_delta()`` to make it clear that they + refer to strided streams. Added missing definition of + ``stream_stride_block()``. +- Changed ``int`` and ``uint`` types in places to use ``ptrdiff_t`` and + ``size_t`` where appropriate. +- Changed API for ``zfp_set_precision()`` and ``zfp_set_accuracy()`` to not + require the scalar type. +- Added missing ``static`` keyword in ``decode_block()``. +- Changed ``testzfp`` to allow specifying which tests to perform on the + command line. +- Modified directory structure. - - Modified directory structure. +**Fixed** - - Added documentation that discusses common issues with using zfp. +- Bug that prevented defining uninitialized arrays. +- Incorrect computation of array sizes in ``zfp_field_size()``. +- Minor issues that prevented code from compiling on Windows. +- Issue with fixed-accuracy headers that caused unnecessary storage. +---- -zfp 0.5.0, February 29, 2016 +0.5.0 (2016-02-29) +------------------ - - Modified CODEC to more efficiently encode blocks whose values are all - zero or are smaller in magnitude than the absolute error tolerance. - This allows representing "empty" blocks using only one bit each. This - version is not backwards compatible with prior zfp versions. +This version introduces backwards incompatible changes to the CODEC. - - Changed behavior of zfp_compress and zfp_decompress to not automatically - rewind the bit stream. This makes it easier to concatenate multiple - compressed bit streams, e.g., when compressing vector fields or multiple - scalars together. +**Added** - - Added functions for compactly encoding the compression parameters - and field meta data, e.g., for producing self-contained compressed - streams. Also added functions for reading and writing a header - containing these parameters. +- Modified CODEC to more efficiently encode blocks whose values are all + zero or are smaller in magnitude than the absolute error tolerance. + This allows representing "empty" blocks using only one bit each. +- Added functions for compactly encoding the compression parameters + and field meta data, e.g., for producing self-contained compressed + streams. Also added functions for reading and writing a header + containing these parameters. - - Changed the zfp example program interface to allow reading and writing - compressed streams, optionally with a header. The zfp tool can now be - used to compress and decompress files as a stand alone utility. +**Changed** +- Changed behavior of ``zfp_compress()`` and ``zfp_decompress()`` to not + automatically rewind the bit stream. This makes it easier to concatenate + multiple compressed bit streams, e.g., when compressing vector fields or + multiple scalars together. +- Changed the |zfp| example program interface to allow reading and writing + compressed streams, optionally with a header. The |zfp| tool can now be + used to compress and decompress files as a stand alone utility. -zfp 0.4.1, December 28, 2015 +---- - - Fixed bug that caused segmentation fault when compressing 3D arrays - whose dimensions are not multiples of four. Specifically, arrays of - dimensions nx * ny * nz, with ny not a multiple of four, were not - handled correctly. +0.4.1 (2015-12-28) +------------------ - - Modified examples/fields.h to ensure standard compliance. Previously, - C99 support was needed to handle the hex float constants, which are - not supported in C++98. +**Added** - - Added simple.c as a minimal example of how to call the compressor. +- Added ``simple.c`` as a minimal example of how to call the compressor. - - Changed compilation of diffusion example to output two executables: - one with and one without compression. +**Changed** +- Changed compilation of diffusion example to output two executables: + one with and one without compression. -zfp 0.4.0, December 5, 2015 +**Fixed** - - Substantial changes to the compression algorithm that improve PSNR - by about 6 dB and speed by a factor of 2-3. These changes are not - backward compatible with previous versions of zfp. +- Bug that caused segmentation fault when compressing 3D arrays whose + dimensions are not multiples of four. Specifically, arrays of dimensions + *nx* |times| *ny* |times| *nz*, with *ny* not a multiple of four, were not + handled correctly. +- Modified ``examples/fields.h`` to ensure standard compliance. Previously, + C99 support was needed to handle the hex float constants, which are + not supported in C++98. - - Added support for 31-bit and 63-bit integer data, as well as shorter - integer types. +---- - - Rewrote compression codec entirely in C to make linking and calling - easier from other programming languages, and to expose the low-level - interface through C instead of C++. This necessitated significant - changes to the API as well. +0.4.0 (2015-12-05) +------------------ - - Minor changes to the C++ compressed array API, as well as major - implementation changes to support the C library. The namespace and - public types are now all in lower case. +This version contains substantial changes to the compression algorithm that +improve PSNR by about 6 dB and speed by a factor of 2-3. These changes are +not backward compatible with previous versions of |zfp|. - - Deprecated support for general fixed-point decorrelating transforms - and slimmed down implementation. +**Added** - - Added new examples for evaluating the throughput of the (de)compressor - and for compressing grayscale images in the pgm format. +- Support for 31-bit and 63-bit integer data, as well as shorter integer types. +- New examples for evaluating the throughput of the (de)compressor and for + compressing grayscale images in the pgm format. +- Frequently asked questions. - - Added FAQ. +**Changed** +- Rewrote compression codec entirely in C to make linking and calling + easier from other programming languages, and to expose the low-level + interface through C instead of C++. This necessitated significant + changes to the API as well. +- Minor changes to the C++ compressed array API, as well as major + implementation changes to support the C library. The namespace and + public types are now all in lower case. -zfp 0.3.2, December 3, 2015 +**Removed** - - Fixed bug in Array::get() that caused the wrong cached block to be - looked up, thus occasionally copying incorrect values back to parts - of the array. +- Support for general fixed-point decorrelating transforms. +---- -zfp 0.3.1, May 6, 2015 +0.3.2 (2015-12-03) +------------------ - - Fixed rare bug caused by exponent underflow in blocks with no normal - and some subnormal numbers. +**Fixed** +- Bug in ``Array::get()`` that caused the wrong cached block to be looked up, + thus occasionally copying incorrect values back to parts of the array. -zfp 0.3.0, March 3, 2015 +---- - - Modified the default decorrelating transform to one that uses only - additions and bit shifts. This new transform, in addition to being - faster, also has some theoretical optimality properties and tends to - improve rate distortion. +0.3.1 (2015-05-06) +------------------ - - Added compile-time support for parameterized transforms, e.g., to - support other popular transforms like DCT, HCT, and Walsh-Hadamard. +**Fixed** - - Made forward transform range preserving: (-1, 1) is mapped to (-1, 1). - Consequently Q1.62 fixed point can be used throughout. +- Rare bug caused by exponent underflow in blocks with no normal and some + subnormal numbers. - - Changed the order in which bits are emitted within each bit plane - to be more intelligent. Group tests are now deferred until they - are needed, i.e., just before the value bits for the group being - tested. This improves the quality of fixed-rate encodings, but - has no impact on compressed size. +---- - - Made several optimizations to improve performance. +0.3.0 (2015-03-03) +------------------ - - Added floating-point traits to reduce the number of template - parameters. It is now possible to declare a 3D array as - Array3, for example. +This version modifies the default decorrelating transform to one that uses +only additions and bit shifts. This new transform, in addition to being +faster, also has some theoretical optimality properties and tends to improve +rate distortion. This change is not backwards compatible. - - Added functions for setting the array scalar type and dimensions. +**Added** - - Consolidated several header files. +- Compile-time support for parameterized transforms, e.g., to support other + popular transforms like DCT, HCT, and Walsh-Hadamard. +- Floating-point traits to reduce the number of template parameters. It is + now possible to declare a 3D array as ``Array3``, for example. +- Functions for setting the array scalar type and dimensions. +- ``testzfp`` for regression testing. - - Added testzfp for regression testing. +**Changed** +- Made forward transform range preserving: (-1, 1) is mapped to (-1, 1). + Consequently Q1.62 fixed point can be used throughout. +- Changed the order in which bits are emitted within each bit plane to be more + intelligent. Group tests are now deferred until they are needed, i.e., just + before the value bits for the group being tested. This improves the quality + of fixed-rate encodings, but has no impact on compressed size. +- Made several optimizations to improve performance. +- Consolidated several header files. -zfp 0.2.1, December 12, 2014 +---- - - Added Win64 support via Microsoft Visual Studio compiler. +0.2.1 (2014-12-12) +------------------ - - Fixed broken support for IBM's xlc compiler. +**Added** - - Made several minor changes to suppress compiler warnings. +- Win64 support via Microsoft Visual Studio compiler. +- Documentation of the expected output for the diffusion example. - - Documented expected output for the diffusion example. +**Changed** +- Made several minor changes to suppress compiler warnings. -zfp 0.2.0, December 2, 2014 +**Fixed** - - The compression interface from zfpcompress was relocated to a - separate library, called libzfp, and modified to be callable from C. - This API now uses a parameter object (zfp_params) to specify array - type and dimensions as well as compression parameters. +- Broken support for IBM's ``xlc`` compiler. - - Several utility functions were added to simplify libzfp usage: +---- - * Functions for setting the rate, precision, and accuracy. - Corresponding functions were also added to the Codec class. +0.2.0 (2014-12-02) +------------------ - * A function for estimating the buffer size needed for compression. +The compression interface from ``zfpcompress`` was relocated to a separate +library, called ``libzfp``, and modified to be callable from C. This API now +uses a parameter object (``zfp_params``) to specify array type and dimensions +as well as compression parameters. - - The Array class functionality was expanded: +**Added** - * Support for accessing the compressed bit stream stored with an - array, e.g., for offline compressed storage and for initializing - an already compressed array. +- Several utility functions were added to simplify ``libzfp`` usage: - * Functions for dynamically specifying the cache size. + * Functions for setting the rate, precision, and accuracy. + Corresponding functions were also added to the ``Codec`` class. + * A function for estimating the buffer size needed for compression. - * The default cache is now direct-mapped instead of two-way - associative. +- The ``Array`` class functionality was expanded: - - Minor bug fixes: + * Support for accessing the compressed bit stream stored with an array, + e.g., for offline compressed storage and for initializing an already + compressed array. + * Functions for dynamically specifying the cache size. + * The default cache is now direct-mapped instead of two-way associative. - * Corrected the value of the lowest possible bit plane to account for - both the smallest exponent and the number of bits in the significand. +**Fixed** - * Corrected inconsistent use of rate and precision. The rate refers - to the number of compressed bits per floating-point value, while - the precision refers to the number of uncompressed bits. The Array - API was changed accordingly. +- Corrected the value of the lowest possible bit plane to account for both + the smallest exponent and the number of bits in the significand. +- Corrected inconsistent use of rate and precision. The rate refers to the + number of compressed bits per floating-point value, while the precision + refers to the number of uncompressed bits. The ``Array`` API was changed + accordingly. +---- -zfp 0.1.0, November 12, 2014 +0.1.0 (2014-11-12) +------------------ - - Initial beta release. +Initial beta release. From 3c54456481d02a7f63c146c4a7a798ff00acb5f6 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 17:21:03 -0700 Subject: [PATCH 116/277] Update authors and contributors --- docs/source/conf.py | 4 ++-- docs/source/contributors.rst | 22 ++++++++++++---------- 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index f6a847a03..b669ef5bf 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -47,7 +47,7 @@ # General information about the project. project = u'zfp' copyright = u'2014-2022, LLNL-CODE-663824' -author = u'Peter Lindstrom' +author = u'Peter Lindstrom, Garrett Morrison' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -149,7 +149,7 @@ # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'zfp.tex', u'zfp Documentation', - u'\shortstack[l]{Peter Lindstrom\\\\Markus Salasoo\\\\Matt Larsen\\\\Stephen Herbein}', 'manual'), + u'\shortstack[l]{Peter Lindstrom\\\\Garrett Morrison}', 'manual'), ] diff --git a/docs/source/contributors.rst b/docs/source/contributors.rst index 5c1067dea..53d33fbf9 100644 --- a/docs/source/contributors.rst +++ b/docs/source/contributors.rst @@ -4,19 +4,21 @@ Contributors ============ -* LLNL |zfp| team +* |zfp| development team - Peter Lindstrom - Garrett Morrison - - Markus Salasoo - - Matt Larsen + +* Major contributors + + - Chuck Atkins - Stephen Herbein + - Mark Kim + - Matt Larsen - Mark Miller + - Markus Salasoo + - David Wade + - Haiying Xu -* External contributors - - - Chuck Atkins, Kitware (CMake support) - - Stephen Hamilton, Johns Hopkins University (VTK plugin) - - Mark Kim, ORNL (original CUDA port) - - Amik St-Cyr, Shell (OpenMP compressor) - - Eric Suchyta, ORNL (ADIOS plugin) +For a full list of contributors, see the +`GitHub Contributors `__ page. From c9ec980f7f9fe261dbb2c398f873a3caabf85982 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 17:22:16 -0700 Subject: [PATCH 117/277] Update zfp homepage URL --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index e760abf31..9e4c39119 100644 --- a/setup.py +++ b/setup.py @@ -6,7 +6,7 @@ version="1.0.0", author="Peter Lindstrom", author_email="zfp@llnl.gov", - url="https://computing.llnl.gov/projects/floating-point-compression", + url="https://zfp.llnl.gov", description="zfp compression in Python", long_description="zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high throughput read and write random access to individual array elements. zfp also supports serial and parallel compression of whole arrays using both lossless and lossy compression with error tolerances. zfp is primarily written in C and C++ but also includes Python and Fortran bindings.", ext_modules=[Extension("zfpy", ["build/python/zfpy.c"], From 43d29affc49970f7afd3c342deeadf7005c883dd Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 18:21:29 -0700 Subject: [PATCH 118/277] Update links to zfp applications --- docs/source/defs.rst | 1 + docs/source/introduction.rst | 12 +++++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/source/defs.rst b/docs/source/defs.rst index 0954f13f4..b0210c1fe 100644 --- a/docs/source/defs.rst +++ b/docs/source/defs.rst @@ -6,6 +6,7 @@ .. |sqrt| unicode:: 0x221a .. |check| unicode:: 0x2713 .. |reg| unicode:: 0x00ae +.. |tm| unicode:: 0x2122 .. |zfp| replace:: zfp .. |cfp| replace:: cfp .. |zforp| replace:: zFORp diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index d0e110430..c2f180f3c 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -72,9 +72,12 @@ Application Support |zfp| has been incorporated into several independently developed applications, plugins, and formats, such as -* `Compressed file I/O `__ +* `Compressed file I/O `__ in `ADIOS `__. +* `Compression codec `__ + in the `BLOSC `__ meta compressor. + * `H5Z-ZFP `__ plugin for `HDF5 `__\ |reg|. |zfp| is also one of the select compressors shipped with @@ -83,10 +86,13 @@ plugins, and formats, such as * `Compression functions `__ for Intel\ |reg| `Integrated Performance Primitives `__. -* `Compressed MPI messages `__ +* `Compressed MPI messages `__ in `MVAPICH2-GDR `__. -* `Compression CODEC `__ +* `Compressed file I/O `__ + in `OpenInventor `__\ |tm|. + +* `Compression codec `__ underlying the `OpenZGY `__ format. From bc29521bee1a2c7a0022e1ed28284635c49a13ee Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 18:21:55 -0700 Subject: [PATCH 119/277] Update authors, zfp homepage URL --- docs/source/license.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/license.rst b/docs/source/license.rst index 4d7cff37c..07c930100 100644 --- a/docs/source/license.rst +++ b/docs/source/license.rst @@ -5,12 +5,12 @@ License | Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC. | Produced at the Lawrence Livermore National Laboratory. -| Written by Peter Lindstrom, Markus Salasoo, Matt Larsen, and Stephen Herbein. +| Written by Peter Lindstrom, Garrett Morrison, Markus Salasoo, Matt Larsen, and Stephen Herbein. | LLNL-CODE-663824. | All rights reserved. This file is part of the zfp library. -For details, see http://computing.llnl.gov/casc/zfp/. +For details, see http://zfp.llnl.gov. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: From 3deba5f1bcf4a17de19279211f457be411f3087a Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 18:49:53 -0700 Subject: [PATCH 120/277] Add Apple clang OpenMP instructions --- docs/source/installation.rst | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 3aec09ceb..b650d7de9 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -131,7 +131,7 @@ Regardless of the settings below, |libzfp| will always be built. .. c:macro:: BUILD_CFP - Build |libcfp| for C bindings to compressed arrays. + Build |libcfp| for C bindings to the compressed-array classes. Default: off. @@ -225,7 +225,9 @@ in the same manner that :ref:`build targets ` are specified, e.g., 0 or OFF to disable OpenMP support. For GNU builds, OpenMP is disabled by default. Set this macro to 1 or ON to enable OpenMP support. See also OMPFLAGS in :file:`Config` in case the compiler does not recognize - :code:`-fopenmp`. NOTE: clang currently does not support OpenMP on macOS. + ``-fopenmp``. For example, Apple clang requires + ``OMPFLAGS=-Xclang -fopenmp``, ``LDFLAGS=-lomp``, and an installation of + ``libomp``. CMake default: on. GNU make default: off. @@ -334,8 +336,9 @@ in the same manner that :ref:`build targets ` are specified, e.g., .. c:macro:: BIT_STREAM_WORD_TYPE Unsigned integer type used for buffering bits. Wider types tend to give - higher performance at the expense of lower bit rate granularity. For - portability of compressed files between little and big endian platforms, + higher performance at the expense of lower + :ref:`bit rate granularity `. For portability of compressed + files between little and big endian platforms, :c:macro:`BIT_STREAM_WORD_TYPE` should be set to :c:type:`uint8`. Default: :c:type:`uint64`. From f88a413192ba8991c76f40e38f97fd238a975627 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 19:29:36 -0700 Subject: [PATCH 121/277] Add variable-length code example --- docs/source/algorithm.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/source/algorithm.rst b/docs/source/algorithm.rst index 826bc6fbd..fd4c72f38 100644 --- a/docs/source/algorithm.rst +++ b/docs/source/algorithm.rst @@ -102,6 +102,12 @@ purposes): |4powd| |minus| *n* bits of the bit plane are run-length encoded as described above, which potentially results in *n* being increased. + As an example, *x* = 000001001101000 with *m* = 10 is encoded as + **0**\ 100\ **1**\ 1\ **1**\ 10\ **1**\ 1000\ **1**, where the bits in + boldface indicate "group tests" that determine if the remainder of *x* + (to the left) contains any one-bits. Again, this variable-length code + is generated and parsed from right to left. + 8. The embedded coder emits one bit at a time, with each successive bit potentially improving the accuracy of the approximation. The early bits are most important and have the greatest impact on accuracy, From e4357d0e2588e6ccd37e8c5e3f6263359bcacfc1 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 20:47:15 -0700 Subject: [PATCH 122/277] Cosmetic changes to compression modes doc --- docs/source/modes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/modes.rst b/docs/source/modes.rst index 087fe04ba..eba15d6bb 100644 --- a/docs/source/modes.rst +++ b/docs/source/modes.rst @@ -58,7 +58,7 @@ the block. The four constraints are as follows: .. c:member:: uint zfp_stream.maxbits The maximum number of bits used to represent a block. This parameter - sets a hard upper bound on compressed block size, and governs the rate + sets a hard upper bound on compressed block size and governs the rate in :ref:`fixed-rate mode `. It may also be used as an upper storage limit to guard against buffer overruns in combination with the accuracy constraints given by :c:member:`zfp_stream.maxprec` and @@ -164,7 +164,7 @@ modes. .. note:: Use fixed-rate mode only if you have to bound the compressed size - or need random access to blocks. + or need read and write random access to blocks. .. _mode-fixed-precision: .. index:: From 46deb61053e0e5590f55bb680a65dce2d6182fde Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 29 Jul 2022 21:07:33 -0700 Subject: [PATCH 123/277] Update execution doc with planned future capabilities --- docs/source/execution.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/source/execution.rst b/docs/source/execution.rst index 8059b0865..614c6f508 100644 --- a/docs/source/execution.rst +++ b/docs/source/execution.rst @@ -295,12 +295,15 @@ fixed storage, and therefore the decompressor needs to be instructed where each compressed block resides in the bit stream to enable parallel decompression. Because the |zfp| bit stream does not currently store such information, variable-rate parallel decompression is not yet -supported. +supported, though plans are to make such functionality available in the +near future. The CUDA implementation supports fixed-rate decompression. OpenMP -fixed-rate decompression will be added in the near future. +fixed-rate decompression has been implemented and will be released in the +near future. Future versions of |zfp| will allow efficient encoding of block sizes and/or offsets to allow each thread to quickly locate the blocks it is responsible for decompressing, which will allow for variable-rate compression and -decompression. +decompression. Such capabilities are already present in the implementation +of the |zfp| :ref:`read-only arrays `. From c6974acec61348eb703235058b5637e3b72e23b1 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 13:31:52 -0700 Subject: [PATCH 124/277] Add zfp_field_blocks() to zFORp --- docs/source/zforp.rst | 10 ++++++++++ fortran/zfp.f90 | 14 ++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/docs/source/zforp.rst b/docs/source/zforp.rst index 5cd0ff4e0..e9e88f645 100644 --- a/docs/source/zforp.rst +++ b/docs/source/zforp.rst @@ -674,6 +674,16 @@ Array Metadata ---- +.. f:function:: zFORp_field_blocks(field) + + Wrapper for :c:func:`zfp_field_blocks` + + :p zFORp_field field [in]: Field metadata + :r blocks: Total number of blocks spanned by field + :rtype blocks: integer (kind=8) + +---- + .. f:function:: zFORp_field_stride(field, stride_arr) Wrapper for :c:func:`zfp_field_stride` diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index b561abbca..c5ae5f1a0 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -399,6 +399,12 @@ function zfp_field_size_bytes(field) result(byte_size) bind(c, name="zfp_field_s integer(c_size_t) :: byte_size end function + function zfp_field_blocks(field) result(blocks) bind(c, name="zfp_field_blocks") + import + type(c_ptr), value :: field + integer(c_size_t) :: blocks + end function + function zfp_field_stride(field, stride_arr) result(is_strided) bind(c, name="zfp_field_stride") import type(c_ptr), value :: field, stride_arr @@ -629,6 +635,7 @@ function zfp_read_header(stream, field, mask) result(num_bits_read) bind(c, name zFORp_field_dimensionality, & zFORp_field_size, & zFORp_field_size_bytes, & + zFORp_field_blocks, & zFORp_field_stride, & zFORp_field_is_contiguous, & zFORp_field_metadata, & @@ -984,6 +991,13 @@ function zFORp_field_size_bytes(field) result(byte_size) bind(c, name="zforp_fie byte_size = zfp_field_size_bytes(field%object) end function zFORp_field_size_bytes + function zFORp_field_blocks(field) result(blocks) bind(c, name="zforp_field_blocks") + implicit none + type(zFORp_field), intent(in) :: field + integer (kind=8) :: blocks + blocks = zfp_field_blocks(field%object) + end function zFORp_field_blocks + function zFORp_field_stride(field, stride_arr) result(is_strided) bind(c, name="zforp_field_stride") implicit none type(zFORp_field), intent(in) :: field From a9203c644f56a9569320308eb54c01b62f2a6ec5 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 14:25:14 -0700 Subject: [PATCH 125/277] Add missing const_array::params() and const_array::set_params() --- include/zfp/constarray1.hpp | 10 ++++++++++ include/zfp/constarray2.hpp | 10 ++++++++++ include/zfp/constarray3.hpp | 10 ++++++++++ include/zfp/constarray4.hpp | 10 ++++++++++ 4 files changed, 40 insertions(+) diff --git a/include/zfp/constarray1.hpp b/include/zfp/constarray1.hpp index ab5c084c7..f2f501de6 100644 --- a/include/zfp/constarray1.hpp +++ b/include/zfp/constarray1.hpp @@ -102,6 +102,9 @@ class const_array1 : public array { // accuracy as absolute error tolerance (fixed-accuracy mode only) double accuracy() const { return store.accuracy(); } + // compression parameters (all compression modes) + void params(uint* minbits, uint* maxbits, uint* maxprec, int* minexp) const { return store.params(minbits, maxbits, maxprec, minexp); } + // set rate in compressed bits per value double set_rate(double rate) { @@ -130,6 +133,13 @@ class const_array1 : public array { store.set_reversible(); } + // set expert mode compression parameters + bool set_params(uint minbits, uint maxbits, uint maxprec, int minexp) + { + cache.clear(); + return store.set_params(minbits, maxbits, maxprec, minexp); + } + // set compression mode and parameters void set_config(const zfp_config& config) { diff --git a/include/zfp/constarray2.hpp b/include/zfp/constarray2.hpp index a14009b42..e89286291 100644 --- a/include/zfp/constarray2.hpp +++ b/include/zfp/constarray2.hpp @@ -105,6 +105,9 @@ class const_array2 : public array { // accuracy as absolute error tolerance (fixed-accuracy mode only) double accuracy() const { return store.accuracy(); } + // compression parameters (all compression modes) + void params(uint* minbits, uint* maxbits, uint* maxprec, int* minexp) const { return store.params(minbits, maxbits, maxprec, minexp); } + // set rate in compressed bits per value double set_rate(double rate) { @@ -133,6 +136,13 @@ class const_array2 : public array { store.set_reversible(); } + // set expert mode compression parameters + bool set_params(uint minbits, uint maxbits, uint maxprec, int minexp) + { + cache.clear(); + return store.set_params(minbits, maxbits, maxprec, minexp); + } + // set compression mode and parameters void set_config(const zfp_config& config) { diff --git a/include/zfp/constarray3.hpp b/include/zfp/constarray3.hpp index 09d755fd9..61d65d466 100644 --- a/include/zfp/constarray3.hpp +++ b/include/zfp/constarray3.hpp @@ -108,6 +108,9 @@ class const_array3 : public array { // accuracy as absolute error tolerance (fixed-accuracy mode only) double accuracy() const { return store.accuracy(); } + // compression parameters (all compression modes) + void params(uint* minbits, uint* maxbits, uint* maxprec, int* minexp) const { return store.params(minbits, maxbits, maxprec, minexp); } + // set rate in compressed bits per value double set_rate(double rate) { @@ -136,6 +139,13 @@ class const_array3 : public array { store.set_reversible(); } + // set expert mode compression parameters + bool set_params(uint minbits, uint maxbits, uint maxprec, int minexp) + { + cache.clear(); + return store.set_params(minbits, maxbits, maxprec, minexp); + } + // set compression mode and parameters void set_config(const zfp_config& config) { diff --git a/include/zfp/constarray4.hpp b/include/zfp/constarray4.hpp index 514e3720f..63680f168 100644 --- a/include/zfp/constarray4.hpp +++ b/include/zfp/constarray4.hpp @@ -111,6 +111,9 @@ class const_array4 : public array { // accuracy as absolute error tolerance (fixed-accuracy mode only) double accuracy() const { return store.accuracy(); } + // compression parameters (all compression modes) + void params(uint* minbits, uint* maxbits, uint* maxprec, int* minexp) const { return store.params(minbits, maxbits, maxprec, minexp); } + // set rate in compressed bits per value double set_rate(double rate) { @@ -139,6 +142,13 @@ class const_array4 : public array { store.set_reversible(); } + // set expert mode compression parameters + bool set_params(uint minbits, uint maxbits, uint maxprec, int minexp) + { + cache.clear(); + return store.set_params(minbits, maxbits, maxprec, minexp); + } + // set compression mode and parameters void set_config(const zfp_config& config) { From 302d0a7348e65e893f0a93f5df7aaafb6ec80fba Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 14:25:55 -0700 Subject: [PATCH 126/277] Fix C89 incompatibility in iteratorC.c --- examples/iteratorC.c | 37 ++++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/examples/iteratorC.c b/examples/iteratorC.c index 1ca909efc..c397e222f 100644 --- a/examples/iteratorC.c +++ b/examples/iteratorC.c @@ -16,7 +16,7 @@ void print2(cfp_ptr2d p, size_t n) const cfp_array2d_api _ = cfp.array2d; while (n--) { - printf("%lf\n", _.reference.get(_.pointer.ref(p))); + printf("%e\n", _.reference.get(_.pointer.ref(p))); p = _.pointer.inc(p); } } @@ -35,12 +35,21 @@ int main() const cfp_array1d_api _1d = cfp.array1d; const cfp_array2d_api _2d = cfp.array2d; const cfp_array3d_api _3d = cfp.array3d; + cfp_array1d v; + cfp_iter1d it1; + cfp_array2d a; + cfp_iter2d it2; + cfp_ptr2d pb2; + cfp_ptr2d pe2; + cfp_array3d b; + cfp_iter3d it3; + cfp_ptr3d pb3; + cfp_ptr3d pe3; size_t i, j, k; /* some fun with 1D arrays */ - cfp_array1d v = _1d.ctor(10, 64.0, 0, 0); + v = _1d.ctor(10, 64.0, 0, 0); /* initialize and print array of random values */ - cfp_iter1d it1; for (it1 = _1d.begin(v); !_1d.iterator.eq(it1, _1d.end(v)); it1 = _1d.iterator.inc(it1)) _1d.reference.set(_1d.iterator.ref(it1), rand()); printf("random array\n"); @@ -48,14 +57,13 @@ int main() printf("\n"); /* some fun with 2D arrays */ - cfp_array2d a = _2d.ctor(5, 7, 64.0, 0, 0); + a = _2d.ctor(5, 7, 64.0, 0, 0); /* print array indices visited in block-order traversal*/ printf("block order (x, y) indices\n"); - cfp_iter2d it2; for (it2 = _2d.begin(a); !_2d.iterator.eq(it2, _2d.end(a)); it2 = _2d.iterator.inc(it2)) { i = _2d.iterator.i(it2); j = _2d.iterator.j(it2); - printf("(%lu, %lu)\n", i, j); + printf("(%lu, %lu)\n", (unsigned long)i, (unsigned long)j); _2d.reference.set(_2d.iterator.ref(it2), i + 10 * j); } printf("\n"); @@ -65,26 +73,25 @@ int main() print2(_2d.ptr_flat(a, 0), _2d.size(a)); printf("\n"); /* pointer arithmetic */ - cfp_ptr2d pb2 = _2d.reference.ptr(_2d.iterator.ref(_2d.begin(a))); - cfp_ptr2d pe2 = _2d.reference.ptr(_2d.iterator.ref(_2d.end(a))); - printf("%lu * %lu = %lld\n", _2d.size_x(a), _2d.size_y(a), (long long int)_2d.pointer.distance(pe2, pb2)); + pb2 = _2d.reference.ptr(_2d.iterator.ref(_2d.begin(a))); + pe2 = _2d.reference.ptr(_2d.iterator.ref(_2d.end(a))); + printf("%lu * %lu = %ld\n", (unsigned long)_2d.size_x(a), (unsigned long)_2d.size_y(a), (long)_2d.pointer.distance(pe2, pb2)); /* some fun with 3D arrays */ - cfp_array3d b = _3d.ctor(7, 2, 5, 64.0, 0, 0); + b = _3d.ctor(7, 2, 5, 64.0, 0, 0); /* print array indices visited in block-order traversal */ printf("block order (x, y, z) indices\n"); - cfp_iter3d it3; for (it3 = _3d.begin(b); !_3d.iterator.eq(it3, _3d.end(b)); it3 = _3d.iterator.inc(it3)) { i = _3d.iterator.i(it3); j = _3d.iterator.j(it3); k = _3d.iterator.k(it3); - printf("(%lu, %lu, %lu)\n", i, j, k); + printf("(%lu, %lu, %lu)\n", (unsigned long)i, (unsigned long)j, (unsigned long)k); } printf("\n"); /* pointer arithmetic */ - cfp_ptr3d pb3 = _3d.reference.ptr(_3d.iterator.ref(_3d.begin(b))); - cfp_ptr3d pe3 = _3d.reference.ptr(_3d.iterator.ref(_3d.end(b))); - printf("%lu * %lu * %lu = %lld\n", _3d.size_x(b), _3d.size_y(b), _3d.size_z(b), (long long int)_3d.pointer.distance(pe3, pb3)); + pb3 = _3d.reference.ptr(_3d.iterator.ref(_3d.begin(b))); + pe3 = _3d.reference.ptr(_3d.iterator.ref(_3d.end(b))); + printf("%lu * %lu * %lu = %ld\n", (unsigned long)_3d.size_x(b), (unsigned long)_3d.size_y(b), (unsigned long)_3d.size_z(b), (long)_3d.pointer.distance(pe3, pb3)); return 0; } From afabe40bbe4e11ceb7850111945f1076e9df2508 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 16:21:57 -0700 Subject: [PATCH 127/277] Add cfp references, pointer, iterators, serialization to CHANGELOG --- CHANGELOG.md | 2 ++ docs/source/versions.rst | 2 ++ 2 files changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0e4f9be92..8b9c05dc2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,8 @@ moved relative to prior versions. - Compressed-array classes for 4D data. - `const` versions of array references, pointers, and iterators. - A more complete API for pointers and iterators. +- cfp support for proxy references and pointers, iterators, and + (de)serialization. - Support for pointers and iterators into array views. - `zfp::array::size_bytes()` allows querying the size of different components of an array object (e.g., payload, cache, index, metadata, ...). diff --git a/docs/source/versions.rst b/docs/source/versions.rst index 018266bdf..e71aa7073 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -19,6 +19,8 @@ moved relative to prior versions. - Compressed-array classes for 4D data. - ``const`` versions of array references, pointers, and iterators. - A more complete API for pointers and iterators. +- |cfp| support for proxy references and pointers, iterators, and + (de)serialization. - Support for pointers and iterators into array views. - ``zfp::array::size_bytes()`` allows querying the size of different components of an array object (e.g., payload, cache, index, metadata, ...). From 49db1371b5e516868f54381277ab03889c97beba Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 17:23:15 -0700 Subject: [PATCH 128/277] Fix incorrect order of distance() arguments --- examples/iteratorC.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/examples/iteratorC.c b/examples/iteratorC.c index c397e222f..8106c493a 100644 --- a/examples/iteratorC.c +++ b/examples/iteratorC.c @@ -8,7 +8,7 @@ void print1(cfp_ptr1d p, size_t n) const cfp_array1d_api _ = cfp.array1d; for (i = 0; i < n; i++) - printf("%e\n", _.reference.get(_.pointer.ref_at(p, i))); + printf("%g\n", _.reference.get(_.pointer.ref_at(p, i))); } void print2(cfp_ptr2d p, size_t n) @@ -16,7 +16,7 @@ void print2(cfp_ptr2d p, size_t n) const cfp_array2d_api _ = cfp.array2d; while (n--) { - printf("%e\n", _.reference.get(_.pointer.ref(p))); + printf("%g\n", _.reference.get(_.pointer.ref(p))); p = _.pointer.inc(p); } } @@ -27,7 +27,7 @@ void print3(cfp_iter1d begin, cfp_iter1d end) cfp_iter1d p; for (p = begin; !_.iterator.eq(p, end); p = _.iterator.inc(p)) - printf("%e\n", _.reference.get(_.iterator.ref(p))); + printf("%g\n", _.reference.get(_.iterator.ref(p))); } int main() @@ -75,7 +75,7 @@ int main() /* pointer arithmetic */ pb2 = _2d.reference.ptr(_2d.iterator.ref(_2d.begin(a))); pe2 = _2d.reference.ptr(_2d.iterator.ref(_2d.end(a))); - printf("%lu * %lu = %ld\n", (unsigned long)_2d.size_x(a), (unsigned long)_2d.size_y(a), (long)_2d.pointer.distance(pe2, pb2)); + printf("%lu * %lu = %ld\n", (unsigned long)_2d.size_x(a), (unsigned long)_2d.size_y(a), (long)_2d.pointer.distance(pb2, pe2)); /* some fun with 3D arrays */ b = _3d.ctor(7, 2, 5, 64.0, 0, 0); @@ -91,7 +91,7 @@ int main() /* pointer arithmetic */ pb3 = _3d.reference.ptr(_3d.iterator.ref(_3d.begin(b))); pe3 = _3d.reference.ptr(_3d.iterator.ref(_3d.end(b))); - printf("%lu * %lu * %lu = %ld\n", (unsigned long)_3d.size_x(b), (unsigned long)_3d.size_y(b), (unsigned long)_3d.size_z(b), (long)_3d.pointer.distance(pe3, pb3)); + printf("%lu * %lu * %lu = %ld\n", (unsigned long)_3d.size_x(b), (unsigned long)_3d.size_y(b), (unsigned long)_3d.size_z(b), (long)_3d.pointer.distance(pb3, pe3)); return 0; } From 8efecf10c8397e288b91db2d21666b3c11521e9a Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 17:46:15 -0700 Subject: [PATCH 129/277] Fix various bugs in diffusionC --- examples/diffusionC.c | 59 +++++++++++++++++++++++-------------------- 1 file changed, 32 insertions(+), 27 deletions(-) diff --git a/examples/diffusionC.c b/examples/diffusionC.c index e6f9ccb77..5e8404564 100644 --- a/examples/diffusionC.c +++ b/examples/diffusionC.c @@ -6,8 +6,8 @@ forward Euler finite difference solution to the heat equation on a 2D grid #include #include #include - #include "zfp/array.h" + #define _ (CFP_NAMESPACE.array2d) #define MAX(x, y) (((nx) > (ny)) ? (nx) : (ny)) @@ -103,9 +103,9 @@ time_step_indexed(double* u, const constants* c) size_t i, x, y; for (y = 1; y < c->ny - 1; y++) for (x = 1; x < c->nx - 1; x++) { - double uxx = (u[y*c->nx + (x - 1)] - 2 * u[y*c->nx + x] + u[y*c->nx + (x + 1)]) / (c->dx * c->dx); - double uyy = (u[(y - 1)*c->nx + x] - 2 * u[y*c->nx + x] + u[(y + 1)*c->nx + x]) / (c->dy * c->dy); - du[y*c->nx + x] = c->dt * c->k * (uxx + uyy); + double uxx = (u[(x - 1) + c->nx * y] - 2 * u[x + c->nx * y] + u[(x + 1) + c->nx * y]) / (c->dx * c->dx); + double uyy = (u[x + c->nx * (y - 1)] - 2 * u[x + c->nx * y] + u[x + c->nx * (y + 1)]) / (c->dy * c->dy); + du[x + c->nx * y] = c->dt * c->k * (uxx + uyy); } /* take forward Euler step */ for (i = 0; i < c->nx * c->ny; i++) @@ -142,7 +142,7 @@ solve(double* u, const constants* c) double t; /* initialize u with point heat source (u is assumed to be zero initialized) */ - u[c->y0*c->nx + c->x0] = 1; + u[c->x0 + c->nx * c->y0] = 1; /* iterate until final time */ for (t = 0; t < c->tfinal; t += c->dt) { @@ -175,7 +175,7 @@ total(const double* u, size_t nx, size_t ny) size_t x, y; for (y = 1; y < ny - 1; y++) for (x = 1; x < nx - 1; x++) - s += u[y*nx + x]; + s += u[x + nx * y]; return s; } @@ -186,9 +186,9 @@ error_compressed(const cfp_array2d u, const constants* c, double t) double e = 0; size_t x, y; for (y = 1; y < c->ny - 1; y++) { - double py = c->dy * (y - c->y0); + double py = c->dy * ((int)y - (int)c->y0); for (x = 1; x < c->nx - 1; x++) { - double px = c->dx * (x - c->x0); + double px = c->dx * ((int)x - (int)c->x0); double f = _.get(u, x, y); double g = c->dx * c->dy * exp(-(px * px + py * py) / (4 * c->k * t)) / (4 * c->pi * c->k * t); e += (f - g) * (f - g); @@ -204,10 +204,10 @@ error(const double* u, const constants* c, double t) double e = 0; size_t x, y; for (y = 1; y < c->ny - 1; y++) { - double py = c->dy * (y - c->y0); + double py = c->dy * ((int)y - (int)c->y0); for (x = 1; x < c->nx - 1; x++) { - double px = c->dx * (x - c->x0); - double f = u[y*c->nx + x]; + double px = c->dx * ((int)x - (int)c->x0); + double f = u[x + c->nx * y]; double g = c->dx * c->dy * exp(-(px * px + py * py) / (4 * c->k * t)) / (4 * c->pi * c->k * t); e += (f - g) * (f - g); } @@ -220,22 +220,23 @@ usage() { fprintf(stderr, "Usage: diffusionC [options]\n"); fprintf(stderr, "Options:\n"); + fprintf(stderr, "-b : use 'blocks' 4x4 blocks of cache\n"); + fprintf(stderr, "-i : traverse arrays using iterators\n"); fprintf(stderr, "-n : number of grid points\n"); + fprintf(stderr, "-r : use compressed arrays with given compressed bits/value\n"); fprintf(stderr, "-t : number of time steps\n"); - fprintf(stderr, "-r : use compressed arrays with 'rate' bits/value\n"); - fprintf(stderr, "-c : use 'blocks' 4x4 blocks of cache\n"); return EXIT_FAILURE; } int main(int argc, char* argv[]) { - int nx = 100; - int ny = 100; + int nx = 128; + int ny = 128; int nt = 0; + int cache_size = 0; double rate = 64; - int iterator = 0; - int compression = 0; - int cache = 0; + zfp_bool iterator = zfp_false; + zfp_bool compression = zfp_false; constants* c = 0; double sum; double err; @@ -246,26 +247,30 @@ int main(int argc, char* argv[]) if (argv[i][0] != '-' || argv[i][2]) return usage(); switch(argv[i][1]) { + case 'b': + if (++i == argc || sscanf(argv[i], "%d", &cache_size) != 1) + return usage(); + cache_size *= (int)(4 * 4 * sizeof(double)); + break; case 'i': - iterator = 1; + iterator = zfp_true; break; case 'n': if (++i == argc || sscanf(argv[i], "%d", &nx) != 1 || ++i == argc || sscanf(argv[i], "%d", &ny) != 1) return usage(); break; - case 't': - if (++i == argc || sscanf(argv[i], "%d", &nt) != 1) - return usage(); - break; case 'r': if (++i == argc || sscanf(argv[i], "%lf", &rate) != 1) return usage(); - compression = 1; + compression = zfp_true; break; - case 'c': - if (++i == argc || sscanf(argv[i], "%d", &cache) != 1) + case 't': + if (++i == argc || sscanf(argv[i], "%d", &nt) != 1) return usage(); + break; + default: + return usage(); } } @@ -274,7 +279,7 @@ int main(int argc, char* argv[]) if (compression) { /* solve problem using compressed arrays */ - cfp_array2d u = _.ctor(nx, ny, rate, 0, cache * 4 * 4 * sizeof(double)); + cfp_array2d u = _.ctor(nx, ny, rate, 0, cache_size); double t = solve_compressed(u, c, iterator); sum = total_compressed(u); err = error_compressed(u, c, t); From 9cc3a25b13b5184d2264fc892ecc60fb37de2bf1 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 18:47:14 -0700 Subject: [PATCH 130/277] Update expected testzfp errors for large arrays --- tests/testzfp.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/testzfp.cpp b/tests/testzfp.cpp index 84a21a840..82b4074ec 100644 --- a/tests/testzfp.cpp +++ b/tests/testzfp.cpp @@ -982,8 +982,8 @@ test(uint dims, ArraySize array_size) }, // large { - {2.441e-04, 7.801e-04, 3.599e-03, 2.793e-02}, - {2.670e-04, 9.075e-04, 3.694e-03, 2.779e-02}, + {2.441e-04, 4.883e-04, 1.222e-03, 2.567e-02}, + {2.670e-04, 4.883e-04, 1.222e-03, 2.567e-02}, } }; double rate = 16; From 6c66c66526d125e162f0be8b39005fa36ebfcd57 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 30 Jul 2022 19:20:09 -0700 Subject: [PATCH 131/277] Proofread and ensure documentation is up to date --- docs/source/arrays.rst | 25 ++++---- docs/source/bit-stream.rst | 14 +++-- docs/source/caching.inc | 6 +- docs/source/cfp.rst | 2 +- docs/source/codec.inc | 2 +- docs/source/directions.rst | 21 +++---- docs/source/disclaimer.inc | 3 +- docs/source/examples.rst | 4 +- docs/source/execution.rst | 1 + docs/source/faq.rst | 25 ++++---- docs/source/high-level-api.rst | 104 +++++++++++++++++---------------- docs/source/index.inc | 6 +- docs/source/installation.rst | 2 +- docs/source/issues.rst | 2 +- docs/source/iterators.inc | 10 ++-- docs/source/limitations.rst | 28 ++++----- docs/source/low-level-api.rst | 27 ++++++--- docs/source/modes.rst | 2 +- docs/source/pointers.inc | 1 + docs/source/references.inc | 10 +++- docs/source/serialization.inc | 12 ++-- docs/source/testing.rst | 2 +- docs/source/tutorial.rst | 60 +++++++++---------- docs/source/views.inc | 11 +++- 24 files changed, 204 insertions(+), 176 deletions(-) diff --git a/docs/source/arrays.rst b/docs/source/arrays.rst index c1c52d45e..74f7dec07 100644 --- a/docs/source/arrays.rst +++ b/docs/source/arrays.rst @@ -163,7 +163,7 @@ in the base class. .. cpp:function:: double array::set_rate(double rate) Set desired compression rate in bits per value. Return the closest rate - supported. See :ref:`FAQ #12 ` and :ref:`FAQ #18 ` + supported. See FAQ :ref:`#12 ` and FAQ :ref:`#18 ` for discussions of the rate granularity. This method destroys the previous contents of the array. @@ -271,28 +271,28 @@ in the base class. .. cpp:function:: iterator array::begin() - Return mutable iterator to beginning of array. + Return random-access mutable iterator to beginning of array. ---- .. cpp:function:: iterator array::end() - Return mutable iterator to end of array. As with STL iterators, the end - points to a virtual element just past the last valid array element. + Return random-access mutable iterator to end of array. As with STL iterators, + the end points to a virtual element just past the last valid array element. ---- .. cpp:function:: const_iterator array::begin() const .. cpp:function:: const_iterator array::cbegin() const - Return const iterator to beginning of array. + Return random-access const iterator to beginning of array. ---- .. cpp:function:: const_iterator array::end() const .. cpp:function:: const_iterator array::cend() const - Return const iterator to end of array. + Return random-access const iterator to end of array. .. note:: Const :ref:`references `, :ref:`pointers `, and @@ -515,10 +515,10 @@ with only a few differences: - Read-only arrays are templated on a block index class that encodes the bit offset to each block of data. Multiple index classes are available - that trade compactness and speed of access. The default index represents - 64-bit offsets using only 16 bits of amortized storage per block. An - "implicit" index is available for fixed-rate read-only arrays, which - computes rather than stores offsets to equal-sized blocks. + that trade compactness and speed of access. The default :cpp:class:`hybrid4` + index represents 64-bit offsets using only 24 bits of amortized storage per + block. An "implicit" index is available for fixed-rate read-only arrays, + which computes rather than stores offsets to equal-sized blocks. .. note:: Whereas variable-rate compression almost always improves accuracy per bit @@ -738,10 +738,11 @@ Additional methods are documented below. ---- -.. cpp:function:: double const_array::set_params(uint minbits, uint maxbits, uint maxprec, int minexp) +.. cpp:function:: bool const_array::set_params(uint minbits, uint maxbits, uint maxprec, int minexp) Set :ref:`expert mode ` parameters. This method destroys the - previous contents of the array. + previous contents of the array. Return whether the codec supports the + combination of parameters. ---- diff --git a/docs/source/bit-stream.rst b/docs/source/bit-stream.rst index e68640238..c77d2ce36 100644 --- a/docs/source/bit-stream.rst +++ b/docs/source/bit-stream.rst @@ -183,7 +183,8 @@ Functions .. c:function:: size_t stream_capacity(const bitstream* stream) - Return byte size of memory buffer associated with *stream*. + Return byte size of memory buffer associated with *stream* specified + in :c:func:`stream_open`. ---- @@ -196,6 +197,7 @@ Functions .. c:function:: uint stream_write_bit(bitstream* stream, uint bit) Write single *bit* to *stream*. *bit* must be one of 0 or 1. + The value of *bit* is returned. ---- @@ -259,9 +261,9 @@ Functions .. c:function:: bitstream_count stream_align(bitstream* stream) - Align stream on next word boundary by skipping bits. No skipping is - done if the stream is already word aligned. Return the number of - skipped bits, if any. + Align stream on next word boundary by skipping bits, i.e., without reading + them. No skipping is done if the stream is already word aligned. Return + the number of skipped bits, if any. ---- @@ -297,5 +299,5 @@ Functions .. c:function:: int stream_set_stride(bitstream* stream, size_t block, ptrdiff_t delta) Set block size, *block*, in number of words and spacing, *delta*, in number - of blocks for strided access. Return nonzero upon success. Requires - :c:macro:`BIT_STREAM_STRIDED`. + of blocks for :ref:`strided access `. Return nonzero upon + success. Requires :c:macro:`BIT_STREAM_STRIDED`. diff --git a/docs/source/caching.inc b/docs/source/caching.inc index 9243b3c5e..374963e01 100644 --- a/docs/source/caching.inc +++ b/docs/source/caching.inc @@ -16,13 +16,13 @@ best choice varies from one application to another, we suggest allocating at least two "layers" of blocks, e.g., 2 |times| (*nx* / 4) |times| (*ny* / 4) blocks for 3D arrays, for applications that stream through the array and perform stencil computations such as gathering data from neighboring elements. -This allows limiting the cache misses to compulsory ones. If the *csize* +This allows limiting the cache misses to compulsory ones. If the *cache_size* parameter provided to the constructor is set to zero bytes, then a default cache size of at least |sqrt|\ *n* blocks is used, where *n* is the total number of blocks contained in the array. -The cache size can be set during construction, or can be set at a later -time via :cpp:func:`array::set_cache_size`. Note that if *csize* = 0, then +The cache size can be set during construction, or can be set at a later time +via :cpp:func:`array::set_cache_size`. Note that if *cache_size* = 0, then the array dimensions must have already been specified for the default size to be computed correctly. When the cache is resized, it is first flushed if not already empty. The cache can also be flushed explicitly if desired diff --git a/docs/source/cfp.rst b/docs/source/cfp.rst index 6d94fdc75..87f474e98 100644 --- a/docs/source/cfp.rst +++ b/docs/source/cfp.rst @@ -795,7 +795,7 @@ and are themselves not modified by these functions. Iterators --------- -|cfp| iterators wrap the C++ :ref:`iterator ` classes. +|cfp| random-access iterators wrap the C++ :ref:`iterator ` classes. All iterators are :ref:`passed by value ` and are themselves not modified by these functions. Iterators are constructed similar to C++ iterators via :c:func:`cfp.array.begin` and diff --git a/docs/source/codec.inc b/docs/source/codec.inc index c806a4923..136e42a39 100644 --- a/docs/source/codec.inc +++ b/docs/source/codec.inc @@ -223,7 +223,7 @@ which provides for 2:1 fixed-rate "compression" using IEEE 754 floating point. ---- .. cpp:function:: size_t codec1::decode_block(bitstream_offset offset, uint shape, Scalar* block) const -.. cpp:function:: size_t codec2::decode_block(bistream_offset offset, uint shape, Scalar* block) const +.. cpp:function:: size_t codec2::decode_block(bitstream_offset offset, uint shape, Scalar* block) const .. cpp:function:: size_t codec3::decode_block(bitstream_offset offset, uint shape, Scalar* block) const .. cpp:function:: size_t codec4::decode_block(bitstream_offset offset, uint shape, Scalar* block) const diff --git a/docs/source/directions.rst b/docs/source/directions.rst index 16c139d42..adff6246c 100644 --- a/docs/source/directions.rst +++ b/docs/source/directions.rst @@ -17,7 +17,7 @@ important features, including: values as missing or indeterminate. Current solutions often rely on tagging missing values as NaNs or special, often very large sentinel values outside the normal range, which can lead to poor compression and complete loss of - accuracy in nearby valid values. See :ref:`FAQ #7 `. + accuracy in nearby valid values. See FAQ :ref:`#7 `. - **Support for NaNs and infinities**. Similar to missing values, some applications store special IEEE floating-point values that are supported @@ -44,7 +44,7 @@ important features, including: at reduced precision over the entire domain, with quality increasing progressively as more data arrives. The low-level bit stream interface already supports progressive access by interleaving bits across blocks - (see :ref:`FAQ #13 `), but |zfp| lacks a high-level API + (see FAQ :ref:`#13 `), but |zfp| lacks a high-level API for generating and accessing progressive streams. - **Parallel compression**. |zfp|'s data partitioning into blocks invites @@ -56,14 +56,15 @@ important features, including: decompression on the GPU via CUDA. However, only fixed-rate mode is so far supported. -- **Variable-rate arrays**. |zfp| currently supports only fixed-rate - compressed arrays, which wastes bits in smooth regions with little - information content while too few bits may be allocated to accurately - preserve sharp features such as shocks and material interfaces, which - tend to drive the physics in numerical simulations. Two candidate - solutions have been identified for read-only and read-write access - to variable-rate arrays with very modest storage overhead. These - arrays will support both fixed precision and accuracy. +- **Variable-rate arrays**. |zfp| currently offers only fixed-rate + compressed arrays with random-access write support; |zfp| |carrrelease| + further provides read-only variable-rate arrays. Fixed-rate arrays waste + bits in smooth regions with little information content while too few bits + may be allocated to accurately preserve sharp features such as shocks and + material interfaces, which tend to drive the physics in numerical + simulations. A candidate solution has been developed for variable-rate + arrays that support read-write random access with modest storage overhead. + We expect to release this capability in the near future. - **Array operations**. |zfp|'s compressed arrays currently support basic indexing and initialization, but lack array-wise operations such as diff --git a/docs/source/disclaimer.inc b/docs/source/disclaimer.inc index 4745f5edc..58a69ba02 100644 --- a/docs/source/disclaimer.inc +++ b/docs/source/disclaimer.inc @@ -5,4 +5,5 @@ map to the innermost (rightmost) array dimension in a C array and to the leftmost dimension in a Fortran array. Getting the order of dimensions right is crucial for good compression and accuracy. See the discussion of - :ref:`dimensions and strides ` for further information. + :ref:`dimensions and strides ` and FAQ :ref:`#0 ` for + further information. diff --git a/docs/source/examples.rst b/docs/source/examples.rst index 786451de1..c73084d0e 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -3,7 +3,7 @@ Code Examples ============= -The :file:`examples` directory includes seven programs that make use of the +The :file:`examples` directory includes ten programs that make use of the compressor. .. _ex-simple: @@ -185,7 +185,7 @@ stream. The program takes one optional argument:: inplace [tolerance] which specifies the fixed-accuracy absolute tolerance to use during -compression. Please see :ref:`FAQ #19 ` for more on the +compression. Please see FAQ :ref:`#19 ` for more on the limitations of in-place compression. .. _ex-iterators: diff --git a/docs/source/execution.rst b/docs/source/execution.rst index 614c6f508..7a69cf2d6 100644 --- a/docs/source/execution.rst +++ b/docs/source/execution.rst @@ -33,6 +33,7 @@ its :ref:`high-level C API `. Parallel compression is not supported via the :ref:`low-level API `, which ignores all execution policy settings and always executes in serial. +.. _exec-policies: Execution Policies ------------------ diff --git a/docs/source/faq.rst b/docs/source/faq.rst index d44a87b15..8d6377bc2 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -141,7 +141,7 @@ of dimensions *nx* |times| *ny*. Can I use a 3D |zfp| array to store this as:: array3d velocity(2, nx, ny, rate); A: Although this could be done, zfp assumes that consecutive values are -related. The two velocity components (*vx*, *vy*) are almost suredly +related. The two velocity components (*vx*, *vy*) are almost assuredly independent and would not be correlated. This will severely hurt the compression rate or quality. Instead, consider storing *vx* and *vy* as two separate 2D scalar arrays:: @@ -501,7 +501,7 @@ Q15: *Must I use the same parameters during compression and decompression?* A: Not necessarily. When decompressing one block at a time, it is possible to use more tightly constrained :c:type:`zfp_stream` parameters during decompression than were used during compression. For instance, one may use a -larger :c:member:`zfp_stream.minbits`, smaller :c:member:`zfp_stream.maxbits`, +smaller :c:member:`zfp_stream.maxbits`, smaller :c:member:`zfp_stream.maxprec`, or larger :c:member:`zfp_stream.minexp` during decompression to process fewer compressed bits than are stored, and to decompress the array more quickly at a lower precision. This may be useful @@ -718,8 +718,8 @@ compression, but with several caveats and restrictions: This is usually easily accomplished in fixed-rate mode, although the expert interface also allows guarding against this in all modes using the :c:member:`zfp_stream.maxbits` parameter. This parameter should be set to - :code:`maxbits = 4^d * 8 * sizeof(type)`, where *d* is the array - dimensionality (1, 2, or 3) and where *type* is the scalar type of the + :code:`maxbits = 4^d * sizeof(type) * 8`, where *d* is the array + dimensionality (1, 2, 3, or 4) and where *type* is the scalar type of the uncompressed data. 4. No header information may be stored in the compressed stream. @@ -1005,7 +1005,7 @@ We recommend experimenting with tolerances and evaluating what error levels are appropriate for each application, e.g., by starting with a low, conservative tolerance and successively doubling it. The distribution of errors produced by |zfp| is approximately Gaussian (see -:ref:`FAQ #30 `), so even if the maximum error may seem large at +:ref:`Q30 `), so even if the maximum error may seem large at an individual grid point, most errors tend to be much smaller and tightly clustered around zero. @@ -1121,7 +1121,7 @@ switching from uncompressed to compressed arrays. Q27: *Do compressed arrays use reference counting?* -A: It is possible to reference compressed array elements via proxy +A: It is possible to reference compressed-array elements via proxy :ref:`references ` and :ref:`pointers `, through :ref:`iterators `, and through :ref:`views `. Such indirect references are valid only during the lifetime of the underlying @@ -1142,10 +1142,11 @@ and not known a priori. The function :c:func:`zfp_stream_maximum_size` returns a buffer size that is guaranteed to be large enough. This function, which should be called *after* setting the desired compression mode and parameters, computes the largest possible compressed data size based on the -current settings and array size. Note that by the pigeonhole principle, any -(lossless) compressor must expand at least one input, so this buffer size may -be larger than the size of the uncompressed input data. :c:func:`zfp_compress` -returns the actual number of bytes of compressed storage. +current compression settings and array size. Note that by the pigeonhole +principle, any (lossless) compressor must expand at least one input, so this +buffer size may be larger than the size of the uncompressed input data. +:c:func:`zfp_compress` returns the actual number of bytes of compressed +storage. When compressing individual blocks using the :ref:`low-level API `, it is useful to know the maximum number of bits that a compressed block @@ -1182,7 +1183,7 @@ Q29: *How can I print array values?* Consider the following seemingly reasonable piece of code:: #include - #include "zfparray1.h" + #include "zfp/array1.hpp" int main() { @@ -1248,7 +1249,7 @@ error distribution is normal and because the worst-case error is often much larger than errors observed in practice, it is common that measured errors are far smaller than the absolute error tolerance specified in :ref:`fixed-accuracy mode ` -(see :ref:`FAQ #22 `). +(see :ref:`Q22 `). It is known that |zfp| errors can be slightly biased and correlated (see :numref:`zfp-rounding` and the third paper above). Recent work has been diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 165beb872..9bfef594a 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -5,15 +5,14 @@ High-Level C API ================ -The C API is broken down into a :ref:`high-level API `, -which handles compression of entire arrays, and a -:ref:`low-level-api ` for processing individual blocks -and managing the underlying bit stream. - -The high-level API should be the API of choice for applications that -compress and decompress entire arrays. A :ref:`low-level API ` -exists for processing individual, possibly partial blocks as well as -reduced-precision integer data less than 32 bits wide. +The |libzfp| C API provides functionality for sequentially compressing and +decompressing whole integer and floating-point arrays or single blocks. It +is broken down into a :ref:`high-level API ` and a +:ref:`low-level API `. The high-level API handles compression of +entire arrays and supports a variety of back-ends (e.g., serial, OpenMP). +The low-level API exists for processing individual, possibly partial blocks +as well as reduced-precision integer data less than 32 bits wide. +Both C APIs are declared in :file:`zfp.h`. The following sections are available: @@ -43,14 +42,23 @@ Macros Macros identifying the |zfp| library version. :c:macro:`ZFP_VERSION` is a single integer constructed from the previous four macros - (see :c:macro:`ZFP_MAKE_VERSION`). :c:macro:`ZFP_VERSION_STRING` is a - string literal (see :c:macro:`ZFP_MAKE_VERSION_STRING`). See also + (see :c:macro:`ZFP_MAKE_VERSION` and :c:macro:`ZFP_MAKE_FULLVERSION`). + :c:macro:`ZFP_VERSION_STRING` is a string literal (see + :c:macro:`ZFP_MAKE_VERSION_STRING`). See also :c:data:`zfp_library_version` and :c:data:`zfp_version_string`. :c:macro:`ZFP_VERSION_TWEAK` is new as of |zfp| |verrelease| and used to mark intermediate develop versions. ---- +.. c:macro:: ZFP_VERSION_DEVELOP + + Macro identifying that the current version is an intermediate develop + version as opposed to an official release. This macro is undefined for + official releases. Available as of |zfp| |verrelease|. + +---- + .. c:macro:: ZFP_MAKE_VERSION(major, minor, patch) .. c:macro:: ZFP_MAKE_VERSION_STRING(major, minor, patch) @@ -67,7 +75,7 @@ Macros Utility macros for constructing :c:macro:`ZFP_VERSION` and :c:macro:`ZFP_VERSION_STRING`, respectively. Includes tweak version - used by intermediate develop versions. Available as of + used by intermediate develop versions. Available as of |zfp| |verrelease|, these macros may be used by applications to test for a certain |zfp| version number, e.g., :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(1, 0, 0, 1)`. @@ -81,14 +89,6 @@ Macros ---- -.. c:macro:: ZFP_DEVELOP - - Macro identifying that the current version is an intermediate develop - version as opposed to an official release. Available as of |zfp| - |verrelease|. - ----- - .. c:macro:: ZFP_MIN_BITS .. c:macro:: ZFP_MAX_BITS .. c:macro:: ZFP_MAX_PREC @@ -138,6 +138,20 @@ for how to read and write header information. Full header information (bitwise OR of all :code:`ZFP_HEADER` constants). +---- + +.. c:macro:: ZFP_MAGIC_BITS +.. c:macro:: ZFP_META_BITS +.. c:macro:: ZFP_MODE_SHORT_BITS +.. c:macro:: ZFP_MODE_LONG_BITS +.. c:macro:: ZFP_HEADER_MAX_BITS +.. c:macro:: ZFP_MODE_SHORT_MAX + + Number of bits used by each portion of the header. These macros are + primarily informational and should not be accessed by the user through + the high-level API. For most common compression parameter settings, + only :c:macro:`ZFP_MODE_SHORT_BITS` bits of header information are stored + to encode the mode (see :c:func:`zfp_stream_mode`). ---- @@ -185,22 +199,9 @@ bitwise ORed together. Use :c:macro:`ZFP_DATA_ALL` to count all storage used. ---- -.. c:macro:: ZFP_MAGIC_BITS -.. c:macro:: ZFP_META_BITS -.. c:macro:: ZFP_MODE_SHORT_BITS -.. c:macro:: ZFP_MODE_LONG_BITS -.. c:macro:: ZFP_HEADER_MAX_BITS -.. c:macro:: ZFP_MODE_SHORT_MAX - - Number of bits used by each portion of the header. These macros are - primarily informational and should not be accessed by the user through - the high-level API. For most common compression parameter settings, - only :c:macro:`ZFP_MODE_SHORT_BITS` bits of header information are stored - to encode the mode (see :c:func:`zfp_stream_mode`). - -.. c:macro: ZFP_ROUND_FIRST -.. c:macro: ZFP_ROUND_NEVER -.. c:macro: ZFP_ROUND_LAST +.. c:macro:: ZFP_ROUND_FIRST +.. c:macro:: ZFP_ROUND_NEVER +.. c:macro:: ZFP_ROUND_LAST Available rounding modes for :c:macro:`ZFP_ROUNDING_MODE`, which specifies at build time how |zfp| performs rounding in lossy compression @@ -240,15 +241,16 @@ Types :: typedef struct { - zfp_exec_policy policy; // execution policy (serial, omp, ...) + zfp_exec_policy policy; // execution policy (serial, omp, cuda, ...) void* params; // execution parameters } zfp_execution; .. warning:: - As of |zfp| |verrelease| `zfp_execution` replaces `zfp_exec_params` with a - ``void *`` to the associated `zfp_exec_params` type (e.g. - `zfp_exec_params_omp`) to limit ABI-breaking changes due to future - extensions to |zfp| execution policies. + As of |zfp| |verrelease| :c:type:`zfp_execution` replaces the former + :code:`zfp_exec_params` with a :code:`void*` to the associated + :code:`zfp_exec_params` type (e.g., :c:type:`zfp_exec_params_omp`) to + limit ABI-breaking changes due to future extensions to |zfp| execution + policies. ---- @@ -270,13 +272,13 @@ Types Execution parameters for OpenMP parallel compression. These are initialized to default values. When nonzero, they indicate the number - of threads to request for parallel compression and the number of 1D - blocks to assign to each thread when compressing 1D arrays. + of threads to request for parallel compression and the number of + consecutive blocks to assign to each thread. :: typedef struct { uint threads; // number of requested threads - uint chunk_size; // number of blocks per chunk (1D only) + uint chunk_size; // number of blocks per chunk } zfp_exec_params_omp; ---- @@ -323,10 +325,10 @@ Types .. c:type:: zfp_type - Enumerates the scalar types supported by the compressor, and is used to - describe the uncompressed array. The compressor and decompressor must use - the same :c:type:`zfp_type`, e.g., one cannot compress doubles and decompress - to floats or integers. + Enumerates the scalar types supported by the compressor and describes the + uncompressed array. The compressor and decompressor must use the same + :c:type:`zfp_type`, e.g., one cannot compress doubles and decompress to + floats or integers. :: typedef enum { @@ -579,7 +581,7 @@ Compression Parameters and indirectly governs the relative error. The actual precision is returned, e.g., in case the desired precision is out of range. To preserve a certain floating-point mantissa or integer precision in the - decompressed data, see :ref:`FAQ #21 `. + decompressed data, see FAQ :ref:`#21 `. ---- @@ -597,7 +599,7 @@ Compression Parameters :ref:`fixed-accuracy mode `. The tolerance ensures that values in the decompressed array differ from the input array by no more than this tolerance (in all but exceptional circumstances; see - :ref:`FAQ #17 `). This compression mode should be used only + FAQ :ref:`#17 `). This compression mode should be used only with floating-point (not integer) data. ---- @@ -612,7 +614,7 @@ Compression Parameters variable-length encoding can be used to economically encode and decode the compression parameters, which is especially important if the parameters are to vary spatially over small regions. Such spatially adaptive coding - would have to be implemented via the low-level API. + would have to be implemented via the :ref:`low-level API `. ---- diff --git a/docs/source/index.inc b/docs/source/index.inc index cf6ba9536..e6e2232f5 100644 --- a/docs/source/index.inc +++ b/docs/source/index.inc @@ -13,9 +13,9 @@ is constant, and the bit offset to each block can be quickly computed. For variable-rate arrays, the compressed block size is data dependent, and additional information must be stored to index the blocks. Toward this end, |zfp| arrays make use of an index class that reports the offset and -size (in number of bits) of each block. The |zfp| :cpp:class:`zfp::array` -and :cpp:class:`zfp::const_array` take such an index class as a template -parameter. This index class is new as of |zfp| |carrrelease|, which +size (in number of bits) of each block. The :cpp:class:`zfp::array` +and :cpp:class:`zfp::const_array` classes take such an index class as a +template parameter. This index class is new as of |zfp| |carrrelease|, which introduced variable-rate arrays. Because |zfp| is designed primarily for very large arrays, the bit offset diff --git a/docs/source/installation.rst b/docs/source/installation.rst index b650d7de9..d43038ce8 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -250,7 +250,7 @@ in the same manner that :ref:`build targets ` are specified, e.g., **Experimental feature**. By default, |zfp| coefficients are truncated, not rounded, which can result in biased errors (see - :ref:`FAQ #30 `). To counter this, two rounding modes are + FAQ :ref:`#30 `). To counter this, two rounding modes are available: :code:`ZFP_ROUND_FIRST` (round during compression; analogous to mid-tread quantization) and :code:`ZFP_ROUND_LAST` (round during decompression; analogous to mid-riser quantization). With diff --git a/docs/source/issues.rst b/docs/source/issues.rst index ecd12d8ee..031be234b 100644 --- a/docs/source/issues.rst +++ b/docs/source/issues.rst @@ -74,7 +74,7 @@ as two scalar fields:: vfield[2][ny][nx] -or by using strides (see also :ref:`FAQ #1 `). Note that in all +or by using strides (see also FAQ :ref:`#1 `). Note that in all these cases |zfp| will still compress the data, but if the dimensionality is not correct then the compression ratio will suffer. diff --git a/docs/source/iterators.inc b/docs/source/iterators.inc index 47235aff1..dc80a71d7 100644 --- a/docs/source/iterators.inc +++ b/docs/source/iterators.inc @@ -28,7 +28,7 @@ consequently, larger compression errors than when the entire block is initialized as a whole. Note that the iterator traversal order differs in this respect from traversal by :ref:`pointers `. -Blocks are visited in raster order similarly to how indidivual array +Blocks are visited in raster order similarly to how individual array elements are indexed, that is, first by *x*, then by *y*, then by *z*, etc. Within each block, elements are visited in the same raster order. All |4powd| values in a block are visited before moving on to the @@ -41,7 +41,8 @@ forward iterators). |zfp| iterators are be used in STL algorithms that support random access iterators. |zfp| |crpirelease| adds :code:`const` qualified versions of iterators, -given by the :code:`const_iterator` class. +given by the :code:`const_iterator` class. Such iterators are available +also for :ref:`read-only arrays `. Per STL mandate, the iterators define several types: @@ -134,8 +135,9 @@ The following operations are defined on iterators: .. cpp:function:: difference_type const_iterator::operator-(const const_iterator& it) const Return difference between this iterator and *it* in number of elements. - The difference *p* |minus| *q* is negative if *p* < *q*. The iterators must - refer to elements in the same array. + The difference *p* |minus| *q* between two iterators, *p* and *q*, is + negative if *p* < *q*. The iterators must refer to elements in the same + array. ---- diff --git a/docs/source/limitations.rst b/docs/source/limitations.rst index eacd6cb5d..fc0dd37cd 100644 --- a/docs/source/limitations.rst +++ b/docs/source/limitations.rst @@ -5,8 +5,8 @@ Limitations =========== -|zfp| has evolved from a research prototype to a library that is approaching -production readiness. However, the API and even the compression codec are +|zfp| has evolved over the years from a research prototype to a production +quality library. However, the API and even the compression codec are still undergoing changes as new important features are added. Below is a list of known limitations of the current version of |zfp|. @@ -47,10 +47,8 @@ that will address some of these limitations. are implemented). These proxy references and pointers can, however, safely be passed to functions and used where regular references and pointers can. -- Although the current version of |zfp| supports :ref:`iterators `, - :ref:`pointers `, and :ref:`references ` to array - elements, 'const' versions of these accessors are not yet available for - read-only access. +- The :ref:`read-only array classes ` do not yet support + (de)serialization. - |zfp| can potentially provide higher precision than conventional float and double arrays, but the interface currently does not expose this. @@ -63,6 +61,8 @@ that will address some of these limitations. instance, compressed 64-bit-per-value storage of 128-bit quad-precision numbers could greatly improve the accuracy of double-precision floating-point computations using the same amount of storage. + The |zfp| compressed-array classes do not yet support integer scalar + types. - Complex-valued arrays are not directly supported. Real and imaginary components must be stored as separate arrays, which may result in lost @@ -77,15 +77,11 @@ that will address some of these limitations. The CUDA implementation is further subject to :ref:`additional limitations `. -- As of version |4drelease|, |zfp| supports compression and decompression - of 4D arrays. However, |zfp| does not yet implement a 4D compressed - array C++ class. This will be added in the near future. - -- The :ref:`C wrappers ` for |zfp|'s compressed arrays support only - a subset of the C++ API. |zfp| |4darrrelease| adds support for proxy +- The |cfp| :ref:`C wrappers ` for |zfp|'s compressed arrays support + only a subset of the C++ API. |zfp| |4darrrelease| adds support for proxy references, pointers, and iterators, but views and read-only arrays are - not yet supported, + not yet supported. Furthermore, |cfp| works only with the |zfp| codec. -- The Python and Fortran bindings do not yet support compressed arrays. - Moreover, only a select subset of the :ref:`high-level API ` - is available via Python. +- The Python and Fortran bindings do not yet support |zfp|'s compressed-array + classes. Moreover, only a select subset of the + :ref:`high-level API ` is available via Python. diff --git a/docs/source/low-level-api.rst b/docs/source/low-level-api.rst index c149c875f..27a2ba7ce 100644 --- a/docs/source/low-level-api.rst +++ b/docs/source/low-level-api.rst @@ -5,14 +5,23 @@ Low-Level C API =============== -The low-level C API provides functionality for compressing individual -*d*-dimensional blocks of up to |4powd| values. If a block is not -complete, i.e., contains fewer than |4powd| values, then |zfp|'s partial +The |libzfp| low-level C API provides functionality for compressing individual +*d*-dimensional blocks of up to |4powd| values. If a block is not complete, +i.e., contains fewer than |4powd| values, then |zfp|'s partial block support should be favored over padding the block with, say, zeros or other fill values. The blocks (de)compressed need not be contiguous and can be gathered from or scattered to a larger array by setting appropriate strides. As of |zfp| |cpprelease|, templated C++ wrappers are also available to simplify calling the low-level API from C++. +The C API is declared in :file:`zfp.h`; the C++ wrappers are found in +:file:`zfp.hpp`. + +.. note:: + Because the unit of parallel work in |zfp| is a *block*, and because the + low-level API operates on individual blocks, this API supports only the + the serial :ref:`execution policy `. Any other execution + policy set in :c:type:`zfp_stream` is silently ignored. For parallel + execution, see the :ref:`high-level API `. The following topics are available: @@ -364,14 +373,14 @@ C++ Wrappers .. cpp:namespace:: zfp To facilitate calling the low-level API from C++, a number of wrappers are -available that are templated on scalar type and dimensionality. Each function -of the form :code:`zfp_function_type_dims`, where *type* denotes scalar type -and *dims* denotes dimensionality, has a corresponding C++ wrapper -:code:`zfp::function`. For example, the C function -:c:func:`zfp_encode_block_float_2` has a C++ wrapper +available (as of |zfp| |cpprelease|) that are templated on scalar type and +dimensionality. Each function of the form :code:`zfp_function_type_dims`, +where *type* denotes scalar type and *dims* denotes dimensionality, has a +corresponding C++ wrapper :code:`zfp::function`. For example, +the C function :c:func:`zfp_encode_block_float_2` has a C++ wrapper :cpp:func:`zfp::encode_block\`. Often *dims* can be inferred from the parameters of overloaded functions, in which case it is omitted as -template parameter. The C++ wrappers are defined in :code:`zfpcpp.h`. +template parameter. The C++ wrappers are defined in :file:`zfp.hpp`. Encoder ^^^^^^^ diff --git a/docs/source/modes.rst b/docs/source/modes.rst index eba15d6bb..4810c6b37 100644 --- a/docs/source/modes.rst +++ b/docs/source/modes.rst @@ -92,7 +92,7 @@ the block. The four constraints are as follows: Note that to achieve a certain accuracy in the decompressed values, the :c:member:`zfp_stream.minexp` value has to be conservatively lowered since |zfp|'s inverse transform may magnify the error (see also - :ref:`FAQs #20-22 `). + FAQs :ref:`#20-22 `). Care must be taken to allow all constraints to be met, as encoding terminates as soon as a single constraint is violated (except diff --git a/docs/source/pointers.inc b/docs/source/pointers.inc index f4d8da956..aceb71ad6 100644 --- a/docs/source/pointers.inc +++ b/docs/source/pointers.inc @@ -60,6 +60,7 @@ and manipulated there, for instance, by passing the pointer by reference via As of |zfp| |crpirelease|, const qualified pointers :code:`const_pointer` are available, and conceptually are equivalent to :code:`const Scalar*`. +Pointers are available for :ref:`read-only arrays ` also. The following operators are defined for proxy pointers. Below *p* refers to the pointer being acted upon. diff --git a/docs/source/references.inc b/docs/source/references.inc index 8834bd9c9..6812b5a24 100644 --- a/docs/source/references.inc +++ b/docs/source/references.inc @@ -36,13 +36,17 @@ take the address of a reference, which yields a :ref:`proxy pointer `. When a reference appears as an rvalue in an expression, it is implicitly converted to a value. -|zfp| |crpirelease| adds :code:`const` qualified versions of references, +|zfp| |crpirelease| adds ``const`` qualified versions of references, pointers, and iterators to support const correctness and potential performance improvements when only read access is needed. As with STL containers, the -corresponding types are prefixed by :code:`const_`, e.g., -:code:`const_reference`. The mutable versions of these classes inherit +corresponding types are prefixed by ``const_``, e.g., +``const_reference``. The mutable versions of these classes inherit the read-only API from the corresponding const versions. +Only references into :ref:`read-write arrays ` are discussed +here; the :ref:`read-only arrays ` support the same +``const_reference`` API. + .. note:: Do not confuse :code:`const_reference` and :code:`const reference`. The former is a reference to an immutable array element, while the latter means diff --git a/docs/source/serialization.inc b/docs/source/serialization.inc index 86928f7c5..7f5da8b51 100644 --- a/docs/source/serialization.inc +++ b/docs/source/serialization.inc @@ -7,8 +7,8 @@ Serialization .. cpp:namespace:: zfp -|zfp|'s compressed arrays can be serialized to sequential, contiguous -storage and later recovered back into an object, e.g., to support +|zfp|'s read-write compressed arrays can be serialized to sequential, +contiguous storage and later recovered back into an object, e.g., to support I/O of compressed-array objects. Two pieces of information are needed to describe a |zfp| array: the raw compressed data, obtained via :cpp:func:`array::compressed_data` and :cpp:func:`array::compressed_size`, @@ -38,10 +38,10 @@ and via a generic :ref:`factory function `: :cpp:func:`array::scalar_type` and :cpp:func:`array::dimensionality`. The (static) factory function is made available by including - :file:`zfpfactory.h`. This header must be included *after* first + :file:`zfp/factory.hpp`. This header must be included *after* first including the header files associated with the compressed arrays, i.e., - :file:`zfparray1.h`, :file:`zfparray2.h`, :file:`zfparray3.h`, and - :file:`zfparray4.h`. Only those arrays whose header files are included + :file:`zfp/array1.hpp`, :file:`zfp/array2.hpp`, :file:`zfp/array3.hpp`, and + :file:`zfp/array4.hpp`. Only those arrays whose header files are included can be constructed by the factory function. This design decouples the array classes so that they may be included independently, for example, to reduce compilation time. @@ -80,6 +80,8 @@ copy:: assert(p->dimensionality() == 3 && p->scalar_type() == zfp_type_double); zfp::array3d& a = *dynamic_cast(p); +When the array is no longer in use, call :code:`delete p;` to deallocate it. + .. note:: The array serialization API changed significantly in |zfp| |crpirelease|. The :cpp:func:`array::get_header` function is now deprecated and has been diff --git a/docs/source/testing.rst b/docs/source/testing.rst index 7d20ab767..cb7e6a608 100644 --- a/docs/source/testing.rst +++ b/docs/source/testing.rst @@ -4,7 +4,7 @@ Regression Tests ================ The :program:`testzfp` program performs basic regression testing by exercising -a small but important subset of :file:`libzfp` and the compressed array +a small but important subset of |libzfp| and the compressed-array classes. It serves as a sanity check that |zfp| has been built properly. These tests assume the default compiler settings, i.e., with none of the settings in :file:`Config` or :file:`CMakeLists.txt` modified. By default, diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst index c1c114516..e43e249ed 100644 --- a/docs/source/tutorial.rst +++ b/docs/source/tutorial.rst @@ -97,7 +97,7 @@ are needed:: // allocate buffer for compressed data size_t bufsize = zfp_stream_maximum_size(zfp, field); - uchar* buffer = new uchar[bufsize]; + void* buffer = malloc(bufsize); Note that :c:func:`zfp_stream_maximum_size` returns the smallest buffer size necessary to safely compress the data---the *actual* compressed size @@ -235,7 +235,7 @@ The block above could also have been compressed as follows using strides:: ptrdiff_t sx = &block[0][0][1] - &block[0][0][0]; // x stride = 1 ptrdiff_t sy = &block[0][1][0] - &block[0][0][0]; // y stride = 4 ptrdiff_t sz = &block[1][0][0] - &block[0][0][0]; // z stride = 16 - size_t bits = zfp_encode_block_strided_double_3(zfp, block, sx, sy, sz); + size_t bits = zfp_encode_block_strided_double_3(zfp, &block[0][0][0], sx, sy, sz); The strides are measured in number of array elements, not in bytes. @@ -281,17 +281,16 @@ Compressed C++ Arrays .. cpp:namespace:: zfp -The |zfp| compressed array API, which currently supports 1D, 2D, and 3D -(but not 4D) arrays, has been designed to facilitate integration with existing -applications. After initial array declaration, a |zfp| array can often -be used in place of a regular C/C++ array or STL vector, e.g., using flat -indexing via :code:`a[index]`, nested indexing :code:`a[k][j][i]` (via -:ref:`nested views `), or using multidimensional indexing via -:code:`a(i)`, :code:`a(i, j)`, or :code:`a(i, j, k)`. There are, -however, some important differences. For instance, applications that -rely on addresses or references to array elements may have to be -modified to use special proxy classes that implement pointers and -references; see :ref:`limitations`. +The |zfp| compressed-array API has been designed to facilitate integration +with existing applications. After initial array declaration, a |zfp| array +can often be used in place of a regular C/C++ array or STL vector, e.g., +using flat indexing via :code:`a[index]`, nested indexing :code:`a[k][j][i]` +(via :ref:`nested views `), or using multidimensional indexing +via :code:`a(i)`, :code:`a(i, j)`, :code:`a(i, j, k)`, or +:code:`a(i, j, k, l)`. There are, however, some important differences. For +instance, applications that rely on addresses or references to array elements +may have to be modified to use special proxy classes that implement pointers +and references; see :ref:`limitations`. |zfp|'s compressed arrays do not support special floating-point values like infinities and NaNs, although subnormal numbers are handled correctly. @@ -305,8 +304,8 @@ The |zfp| C++ classes are implemented entirely as header files and make extensive use of C++ templates to reduce code redundancy. These classes are wrapped in the :cpp:any:`zfp` namespace. -Currently, there are six array classes for 1D, 2D, and 3D arrays, each of -which can represent single- or double-precision values. Although these +Currently, there are eight array classes for 1D, 2D, 3D, and 4D arrays, each +of which can represent single- or double-precision values. Although these arrays store values in a form different from conventional single- and double-precision floating point, the user interacts with the arrays via floats and doubles. @@ -346,6 +345,7 @@ increments of 64 / |4powd| bits in *d* dimensions, i.e. 1D arrays: 16-bit granularity 2D arrays: 4-bit granularity 3D arrays: 1-bit granularity + 4D arrays: 1/4-bit granularity For finer granularity, the :c:macro:`BIT_STREAM_WORD_TYPE` macro needs to be set to a type narrower than 64 bits during compilation of |libzfp|, @@ -356,6 +356,7 @@ bits in *d* dimensions, or 1D arrays: 2-bit granularity 2D arrays: 1/2-bit granularity 3D arrays: 1/8-bit granularity + 4D arrays: 1/32-bit granularity Note that finer granularity usually implies slightly lower performance. Also note that because the arrays are stored compressed, their effective @@ -403,7 +404,7 @@ directly without having to convert to/from its floating-point representation:: The array can through this pointer be initialized from offline compressed storage, but only after its dimensions and rate have been specified (see -above). For this to work properly, the cache must first be emptied via a +above). For this to work properly, the cache must first be emptied via an :cpp:func:`array::clear_cache` call (see below). Through operator overloading, the array can be accessed in one of two ways. @@ -497,14 +498,14 @@ for references and pointers that guarantee persistent access by referencing elements by array object and index. These classes perform decompression on demand, much like how Boolean vector references are implemented in the STL. -Iterators for 1D arrays support random access, while 2D and 3D array iterators -are merely forward (sequential) iterators. All iterators ensure that array -values are visited one block at a time, and are the preferred way of looping -over array elements. Such block-by-block access is especially useful when -performing write accesses since then complete blocks are updated one at a -time, thus reducing the likelihood of a partially updated block being evicted -from the cache and compressed, perhaps with some values in the block being -uninitialized. Here is an example of initializing a 3D array:: +As of |zfp| |raiterrelease|, all iterators for 1D-4D arrays support random +access. Iterators ensure that array values are visited one block at a time, +and are the preferred way of looping over array elements. Such block-by-block +access is especially useful when performing write accesses since then complete +blocks are updated one at a time, thus reducing the likelihood of a partially +updated block being evicted from the cache and compressed, perhaps with some +values in the block being uninitialized. Here is an example of initializing +a 3D array:: for (zfp::array3d::iterator it = a.begin(); it != a.end(); it++) { size_t i = it.i(); @@ -553,9 +554,8 @@ row-major order, i.e. where :code:`&a(i, j, k)` and :code:`&a[0]` are both of type :cpp:class:`array3d::pointer`. Thus, iterators and pointers do not visit arrays in the same order, except for the special case of 1D arrays. -Unlike iterators, pointers support random access for arrays of all -dimensions and behave very much like :code:`float*` and :code:`double*` -built-in pointers. +Like iterators, pointers support random access for arrays of all dimensions +and behave very much like :code:`float*` and :code:`double*` built-in pointers. Proxy objects for array element references have been supported since the first release of |zfp|, and may for instance be used in place of @@ -591,9 +591,9 @@ at least two layers of blocks (2 |times| (*nx* / 4) |times| (*ny* / 4) blocks) for applications that stream through the array and perform stencil computations such as gathering data from neighboring elements. This allows limiting the cache misses to compulsory ones. If the *cache_size* parameter -is set to zero bytes, then a default size of |sqrt|\ *n* blocks is used, -where *n* is the total number of blocks in the array. - +is set to zero bytes, then a default size of |sqrt|\ *n* blocks (rounded +up to the next integer power of two) is used, where *n* is the total number +of blocks in the array. The cache size can be set during construction, or can be set at a later time via diff --git a/docs/source/views.inc b/docs/source/views.inc index e5e081df0..5c1364e09 100644 --- a/docs/source/views.inc +++ b/docs/source/views.inc @@ -54,6 +54,11 @@ iterators. traversed using pointers and iterators. We have :code:`view(10, 7) == (&view(0, 0))[87] == view.begin()[97] == view.end()[-2]`. +With the |zfp| |carrrelease| release of +:ref:`read-only arrays `, such arrays also support the two +kinds of immutable views (:code:`const_view` and :code:`private_const_view`). +The documentation below applies to views into read-only arrays as well. + .. note:: Like iterators and proxy references and pointers, a view is valid only during the lifetime of the array that it references. **No reference @@ -192,7 +197,7 @@ the arrays that they reference. .. cpp:function:: array3::const_view::const_iterator array3::const_view::cbegin() const .. cpp:function:: array4::const_view::const_iterator array4::const_view::cbegin() const - Const iterator to first element of view. + Random-access const iterator to first element of view. ---- @@ -205,7 +210,7 @@ the arrays that they reference. .. cpp:function:: array3::const_view::const_iterator array3::const_view::cend() const .. cpp:function:: array4::const_view::const_iterator array4::const_view::cend() const - Const iterator to end of view. + Random-access const iterator to end of view. There are a number of common methods inherited from a base class, :code:`preview`, further up the class hierarchy. @@ -579,7 +584,7 @@ nested view to in effect provide nested array indexing:: .. cpp:function:: reference array4::nested_view3::operator()(size_t i, size_t j, size_t k) .. cpp:function:: reference array4::nested_view4::operator()(size_t i, size_t j, size_t k, size_t l) - Return reference to a scalar element of a 2D or 3D array. + Return reference to a scalar element of a 2D, 3D, or 4D array. .. _slicing: From 3ed74345d0e4644a7cc3aff49dc7a53e5a344498 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 31 Jul 2022 10:39:52 -0700 Subject: [PATCH 132/277] Ensure make clean removes libcfp --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index db433ab23..aacf77894 100644 --- a/Makefile +++ b/Makefile @@ -33,7 +33,7 @@ test: # clean all clean: @cd src; $(MAKE) clean - @cd src; $(MAKE) clean + @cd cfp; $(MAKE) clean @cd fortran; $(MAKE) clean @cd utils; $(MAKE) clean @cd tests; $(MAKE) clean From 2445a7c3ca2f6eb2e860b7739b33dd1d4dc29594 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 31 Jul 2022 10:41:10 -0700 Subject: [PATCH 133/277] Bump zFORp_library_version --- fortran/zfp.f90 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index c5ae5f1a0..b4b776404 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -65,7 +65,7 @@ module zfp integer, protected, bind(c, name="zFORp_codec_version") :: zFORp_codec_version data zFORp_codec_version/const_zFORp_codec_version/ - integer, parameter :: const_zFORp_library_version = 85 ! 0x55 + integer, parameter :: const_zFORp_library_version = 4096 ! 0x1000 integer, protected, bind(c, name="zFORp_library_version") :: zFORp_library_version data zFORp_library_version/const_zFORp_library_version/ From e1af8487bd6a9610a1990448032f35f8c7b69593 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 31 Jul 2022 10:42:32 -0700 Subject: [PATCH 134/277] Bump copyright date, add copyright notice to zfp.hpp --- include/zfp.h | 2 +- include/zfp.hpp | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/include/zfp.h b/include/zfp.h index f4a3f66f5..9db4a3873 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -1,5 +1,5 @@ /* -** Copyright (c) 2014-2019, Lawrence Livermore National Security, LLC and +** Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC and ** other zfp project contributors. See the top-level LICENSE file for details. ** SPDX-License-Identifier: BSD-3-Clause */ diff --git a/include/zfp.hpp b/include/zfp.hpp index b215ac0de..406fec9bb 100644 --- a/include/zfp.hpp +++ b/include/zfp.hpp @@ -1,6 +1,10 @@ #ifndef ZFP_HPP #define ZFP_HPP +// Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC and +// other zfp project contributors. See the top-level LICENSE file for details. +// SPDX-License-Identifier: BSD-3-Clause + #include "zfp.h" // templated C++ wrappers around libzfp low-level C functions From 07485e1a39f00c1a499544a598e7959442023d82 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 31 Jul 2022 18:08:13 -0700 Subject: [PATCH 135/277] Update ReadTheDocs badge link --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 50fa4595e..93726145e 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ZFP === [![Github Actions Build Status](https://github.com/LLNL/zfp/actions/workflows/main.yml/badge.svg?branch=develop)](https://github.com/LLNL/zfp/actions/workflows/main.yml) [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) -[![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release0.5.5)](https://zfp.readthedocs.io/en/release0.5.5/?badge=release0.5.5) +[![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.0)](https://zfp.readthedocs.io/en/release1.0.0/?badge=release1.0.0) [![Code Coverage](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg)](https://codecov.io/gh/LLNL/zfp) zfp is a compressed format for representing multidimensional floating-point From fff64a14a703ac3b1708129098b62d1bdb0bfae8 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 31 Jul 2022 18:36:36 -0700 Subject: [PATCH 136/277] Revert to gencodec for consistency with zfpcodec --- examples/diffusion.cpp | 2 +- include/zfp/codec/{generic.hpp => gencodec.hpp} | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) rename include/zfp/codec/{generic.hpp => gencodec.hpp} (99%) diff --git a/examples/diffusion.cpp b/examples/diffusion.cpp index 08cf66949..a62f191e1 100644 --- a/examples/diffusion.cpp +++ b/examples/diffusion.cpp @@ -9,7 +9,7 @@ #include #include "zfp/array2.hpp" #include "zfp/constarray2.hpp" -#include "zfp/codec/generic.hpp" +#include "zfp/codec/gencodec.hpp" #include "array2d.hpp" // add half precision if compiler supports it diff --git a/include/zfp/codec/generic.hpp b/include/zfp/codec/gencodec.hpp similarity index 99% rename from include/zfp/codec/generic.hpp rename to include/zfp/codec/gencodec.hpp index 6f89183a7..b0eb3230f 100644 --- a/include/zfp/codec/generic.hpp +++ b/include/zfp/codec/gencodec.hpp @@ -16,7 +16,6 @@ #include #include #include "zfp.h" -#include "zfp/array.hpp" #include "zfp/internal/array/memory.hpp" #include "zfp/internal/array/traits.hpp" From 9ac63801cc6070b2f8532bb8429a84b301b2f5c5 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Aug 2022 10:01:38 -0700 Subject: [PATCH 137/277] Document change in ZFP_VERSION mapping --- docs/source/high-level-api.rst | 56 ++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 13 deletions(-) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 9bfef594a..cc3b59127 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -37,34 +37,64 @@ Macros .. c:macro:: ZFP_VERSION_MINOR .. c:macro:: ZFP_VERSION_PATCH .. c:macro:: ZFP_VERSION_TWEAK -.. c:macro:: ZFP_VERSION -.. c:macro:: ZFP_VERSION_STRING - Macros identifying the |zfp| library version. :c:macro:`ZFP_VERSION` is - a single integer constructed from the previous four macros - (see :c:macro:`ZFP_MAKE_VERSION` and :c:macro:`ZFP_MAKE_FULLVERSION`). - :c:macro:`ZFP_VERSION_STRING` is a string literal (see - :c:macro:`ZFP_MAKE_VERSION_STRING`). See also - :c:data:`zfp_library_version` and :c:data:`zfp_version_string`. - :c:macro:`ZFP_VERSION_TWEAK` is new as of |zfp| |verrelease| and used - to mark intermediate develop versions. + Macros identifying the |zfp| library version + (*major*.\ *minor*.\ *patch*.\ *tweak*). :c:macro:`ZFP_VERSION_TWEAK` + is new as of |zfp| |verrelease| and is used to mark intermediate develop + versions (unofficial releases). ---- .. c:macro:: ZFP_VERSION_DEVELOP Macro identifying that the current version is an intermediate develop - version as opposed to an official release. This macro is undefined for + version and not an official release. This macro is undefined for official releases. Available as of |zfp| |verrelease|. ---- +.. c:macro:: ZFP_VERSION + + A single integer constructed from the four version identifiers + :c:macro:`ZFP_VERSION_MAJOR`, + :c:macro:`ZFP_VERSION_MINOR`, + :c:macro:`ZFP_VERSION_PATCH`, and + :c:macro:`ZFP_VERSION_TWEAK`. + This integer can be generated by :c:macro:`ZFP_MAKE_VERSION` or + :c:macro:`ZFP_MAKE_FULLVERSION`. Its value equals the global constant + :c:data:`zfp_library_version`. + +.. note:: + Although :c:macro:`ZFP_VERSION` increases monotonically with release date + and with the four version identifiers it depends on, the mapping + to :c:macro:`ZFP_VERSION` changed with the introduction of + :c:macro:`ZFP_VERSION_TWEAK` in |zfp| |verrelease|. + + Going forward, we recommend using :c:macro:`ZFP_MAKE_VERSION` or + :c:macro:`ZFP_MAKE_FULLVERSION` in conditional code that depends on + :c:macro:`ZFP_VERSION`, e.g., + :code:`#if ZFP_VERSION >= ZFP_MAKE_VERSION(1, 0, 0)`. + Note that such constructions should not be used with older versions of + |zfp|, e.g., :code:`if (zfp_library_version == ZFP_MAKE_VERSION(0, 5, 5))` + will not give the expected result with binary versions of |libzfp| before + version |verrelease|. + +---- + +.. c:macro:: ZFP_VERSION_STRING + + :c:macro:`ZFP_VERSION_STRING` is a string literal composed of the four + version identifiers and release date suitable for printing. It equals + :c:data:`zfp_version_string`. + +---- + .. c:macro:: ZFP_MAKE_VERSION(major, minor, patch) .. c:macro:: ZFP_MAKE_VERSION_STRING(major, minor, patch) Utility macros for constructing :c:macro:`ZFP_VERSION` and :c:macro:`ZFP_VERSION_STRING`, respectively. Available as of - |zfp| |64bitrelease|, these macros may be used by applications to test + |zfp| |verrelease|, these macros may be used by applications to test for a certain |zfp| version number, e.g., :code:`#if ZFP_VERSION >= ZFP_MAKE_VERSION(1, 0, 0)`. @@ -78,7 +108,7 @@ Macros used by intermediate develop versions. Available as of |zfp| |verrelease|, these macros may be used by applications to test for a certain |zfp| version number, e.g., - :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(1, 0, 0, 1)`. + :code:`#if ZFP_VERSION >= ZFP_MAKE_FULLVERSION(1, 0, 0, 2)`. ---- From ed01090c82e3763e4d064fd53628eda03a2efc39 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Aug 2022 10:15:27 -0700 Subject: [PATCH 138/277] Add version_tweak to zFORp --- docs/source/zforp.rst | 6 ++++++ fortran/zfp.f90 | 10 +++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/source/zforp.rst b/docs/source/zforp.rst index e9e88f645..aee51715a 100644 --- a/docs/source/zforp.rst +++ b/docs/source/zforp.rst @@ -115,6 +115,12 @@ Non-Enum Constants ---- +.. f:variable:: integer zFORp_version_tweak + + Wraps :c:macro:`ZFP_VERSION_TWEAK` + +---- + .. f:variable:: integer zFORp_codec_version Wraps :c:data:`zfp_codec_version` diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index b4b776404..f671d1449 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -54,12 +54,15 @@ module zfp integer, parameter :: const_zFORp_version_major = 1 integer, parameter :: const_zFORp_version_minor = 0 integer, parameter :: const_zFORp_version_patch = 0 + integer, parameter :: const_zFORp_version_tweak = 0 integer, protected, bind(c, name="zFORp_version_major") :: zFORp_version_major integer, protected, bind(c, name="zFORp_version_minor") :: zFORp_version_minor integer, protected, bind(c, name="zFORp_version_patch") :: zFORp_version_patch + integer, protected, bind(c, name="zFORp_version_tweak") :: zFORp_version_tweak data zFORp_version_major/const_zFORp_version_major/, & zFORp_version_minor/const_zFORp_version_minor/, & - zFORp_version_patch/const_zFORp_version_patch/ + zFORp_version_patch/const_zFORp_version_patch/, & + zFORp_version_tweak/const_zFORp_version_tweak/ integer, parameter :: const_zFORp_codec_version = 5 integer, protected, bind(c, name="zFORp_codec_version") :: zFORp_codec_version @@ -69,7 +72,7 @@ module zfp integer, protected, bind(c, name="zFORp_library_version") :: zFORp_library_version data zFORp_library_version/const_zFORp_library_version/ - character(len = 36), parameter :: zFORp_version_string = 'zfp version 1.0.0 (July XX, 2022)' + character(len = 36), parameter :: zFORp_version_string = 'zfp version 1.0.0 (August 1, 2022)' integer, parameter :: const_zFORp_min_bits = 1 integer, parameter :: const_zFORp_max_bits = 16658 @@ -548,7 +551,8 @@ function zfp_read_header(stream, field, mask) result(num_bits_read) bind(c, name ! C macros -> constants public :: zFORp_version_major, & zFORp_version_minor, & - zFORp_version_patch + zFORp_version_patch, & + zFORp_version_tweak public :: zFORp_codec_version, & zFORp_library_version, & From 1fc3539e919c66f8e7026d04652f98f43ba9a97e Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Aug 2022 10:17:00 -0700 Subject: [PATCH 139/277] Align license.rst with LICENSE --- docs/source/license.rst | 47 +++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 25 deletions(-) diff --git a/docs/source/license.rst b/docs/source/license.rst index 07c930100..0c8d6e5ce 100644 --- a/docs/source/license.rst +++ b/docs/source/license.rst @@ -3,15 +3,9 @@ License ======= -| Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC. -| Produced at the Lawrence Livermore National Laboratory. -| Written by Peter Lindstrom, Garrett Morrison, Markus Salasoo, Matt Larsen, and Stephen Herbein. -| LLNL-CODE-663824. +| Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC | All rights reserved. -This file is part of the zfp library. -For details, see http://zfp.llnl.gov. - Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: @@ -39,24 +33,27 @@ ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -Additional BSD Notice +Notice +------ + +This work was produced under the auspices of the U.S. Department of +Energy by Lawrence Livermore National Laboratory under Contract +DE-AC52-07NA27344. -1. This notice is required to be provided under our contract with the U.S. -Department of Energy (DOE). This work was produced at Lawrence Livermore -National Laboratory under Contract No. DE-AC52-07NA27344 with the DOE. +This work was prepared as an account of work sponsored by an agency of +the United States Government. Neither the United States Government nor +Lawrence Livermore National Security, LLC, nor any of their employees +makes any warranty, expressed or implied, or assumes any legal liability +or responsibility for the accuracy, completeness, or usefulness of any +information, apparatus, product, or process disclosed, or represents that +its use would not infringe privately owned rights. -2. Neither the United States Government nor Lawrence Livermore National -Security, LLC nor any of their employees, makes any warranty, express or -implied, or assumes any liability or responsibility for the accuracy, -completeness, or usefulness of any information, apparatus, product, or -process disclosed, or represents that its use would not infringe -privately-owned rights. +Reference herein to any specific commercial product, process, or service +by trade name, trademark, manufacturer, or otherwise does not necessarily +constitute or imply its endorsement, recommendation, or favoring by the +United States Government or Lawrence Livermore National Security, LLC. -3. Also, reference herein to any specific commercial products, process, or -services by trade name, trademark, manufacturer or otherwise does not -necessarily constitute or imply its endorsement, recommendation, or -favoring by the United States Government or Lawrence Livermore National -Security, LLC. The views and opinions of authors expressed herein do not -necessarily state or reflect those of the United States Government or -Lawrence Livermore National Security, LLC, and shall not be used for -advertising or product endorsement purposes. +The views and opinions of authors expressed herein do not necessarily +state or reflect those of the United States Government or Lawrence +Livermore National Security, LLC, and shall not be used for advertising +or product endorsement purposes. From 074f01092bddfdc7763ce4e1fae25f1c1e48ac53 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Aug 2022 11:07:45 -0700 Subject: [PATCH 140/277] Point badges and documentation to release1.0.0 branch --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 93726145e..8f207f162 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,8 @@ ZFP === -[![Github Actions Build Status](https://github.com/LLNL/zfp/actions/workflows/main.yml/badge.svg?branch=develop)](https://github.com/LLNL/zfp/actions/workflows/main.yml) -[![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) -[![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.0)](https://zfp.readthedocs.io/en/release1.0.0/?badge=release1.0.0) -[![Code Coverage](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg)](https://codecov.io/gh/LLNL/zfp) +[![Github Actions Build Status](https://github.com/LLNL/zfp/actions/workflows/main.yml/badge.svg?branch=release1.0.0)](https://github.com/LLNL/zfp/actions/workflows/main.yml) +[![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/release1.0.0?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) +[![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.0)](https://zfp.readthedocs.io/en/release1.0.0/) zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high @@ -56,18 +55,19 @@ Note: GNU builds are less flexible and do not support all available features, e.g., CUDA support. For further configuration and build instructions, please consult the -[documentation](https://zfp.readthedocs.io/en/latest/installation.html). +[documentation](https://zfp.readthedocs.io/en/release1.0.0/installation.html). For examples of how to call the C library and use the C++ array classes, -see the [examples](https://zfp.readthedocs.io/en/latest/examples.html) +see the [examples](https://zfp.readthedocs.io/en/release1.0.0/examples.html) section. Documentation ------------- -Full HTML [documentation](http://zfp.readthedocs.io/) is available online. -A [PDF](http://readthedocs.org/projects/zfp/downloads/pdf/latest/) version -is also available. +Full HTML [documentation](http://zfp.readthedocs.io/en/release1.0.0) is +available online. +A [PDF](http://readthedocs.org/projects/zfp/downloads/pdf/release1.0.0/) +version is also available. Further information on the zfp software is included in these files: From 7f84893b1d52ed2c2506601ba97fc70310036032 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Aug 2022 11:29:14 -0700 Subject: [PATCH 141/277] Clarify ZFP_VERSION_DEVELOP meaning and value --- docs/source/high-level-api.rst | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index cc3b59127..f47732869 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -33,6 +33,8 @@ The following sections are available: Macros ------ +.. _version-id: + .. c:macro:: ZFP_VERSION_MAJOR .. c:macro:: ZFP_VERSION_MINOR .. c:macro:: ZFP_VERSION_PATCH @@ -47,27 +49,25 @@ Macros .. c:macro:: ZFP_VERSION_DEVELOP - Macro identifying that the current version is an intermediate develop - version and not an official release. This macro is undefined for - official releases. Available as of |zfp| |verrelease|. + Macro signifying that the current version is an intermediate version that + differs from the last official release. This macro is undefined for + official releases; when defined, its value equals 1. Note that this + macro may be defined even if the four :ref:`version identifiers ` + have not changed. Available as of |zfp| |verrelease|. ---- .. c:macro:: ZFP_VERSION - A single integer constructed from the four version identifiers - :c:macro:`ZFP_VERSION_MAJOR`, - :c:macro:`ZFP_VERSION_MINOR`, - :c:macro:`ZFP_VERSION_PATCH`, and - :c:macro:`ZFP_VERSION_TWEAK`. - This integer can be generated by :c:macro:`ZFP_MAKE_VERSION` or - :c:macro:`ZFP_MAKE_FULLVERSION`. Its value equals the global constant - :c:data:`zfp_library_version`. + A single integer constructed from the four + :ref:`version identifiers `. This integer can be generated by + :c:macro:`ZFP_MAKE_VERSION` or :c:macro:`ZFP_MAKE_FULLVERSION`. Its value + equals the global constant :c:data:`zfp_library_version`. .. note:: Although :c:macro:`ZFP_VERSION` increases monotonically with release date - and with the four version identifiers it depends on, the mapping - to :c:macro:`ZFP_VERSION` changed with the introduction of + and with the four :ref:`version identifiers ` it depends on, + the mapping to :c:macro:`ZFP_VERSION` changed with the introduction of :c:macro:`ZFP_VERSION_TWEAK` in |zfp| |verrelease|. Going forward, we recommend using :c:macro:`ZFP_MAKE_VERSION` or @@ -84,7 +84,7 @@ Macros .. c:macro:: ZFP_VERSION_STRING :c:macro:`ZFP_VERSION_STRING` is a string literal composed of the four - version identifiers and release date suitable for printing. It equals + :ref:`version identifiers `. It is a component of :c:data:`zfp_version_string`. ---- From 82a557c82c353cd29e562c001e449ccc1a190bf5 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Aug 2022 11:30:19 -0700 Subject: [PATCH 142/277] Set release version and date --- CHANGELOG.md | 2 +- docs/source/versions.rst | 2 +- src/zfp.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b9c05dc2..3251d5155 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ Change Log --- -## 1.0.0 (2022-07-XX) +## 1.0.0 (2022-08-01) This release is not ABI compatible with prior releases due to numerous changes to function signatures and data structures like `zfp_field`. However, few of diff --git a/docs/source/versions.rst b/docs/source/versions.rst index e71aa7073..d246adbbf 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -3,7 +3,7 @@ Release Notes ============= -1.0.0 (2022-07-XX) +1.0.0 (2022-08-01) ------------------ This release is not ABI compatible with prior releases due to numerous changes diff --git a/src/zfp.c b/src/zfp.c index 736c1f857..a498a985f 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -11,7 +11,7 @@ const uint zfp_codec_version = ZFP_CODEC; const uint zfp_library_version = ZFP_VERSION; -const char* const zfp_version_string = "zfp version " ZFP_VERSION_STRING " (July XX, 2022)"; +const char* const zfp_version_string = "zfp version " ZFP_VERSION_STRING " (August 1, 2022)"; /* private functions ------------------------------------------------------- */ From fdc30ff6984430e8fbac6a9fa6eff93af9bc3504 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 10 Aug 2022 13:11:57 -0700 Subject: [PATCH 143/277] Set ZFP_VERSION_DEVELOP --- include/zfp/version.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/zfp/version.h b/include/zfp/version.h index 790927f30..75872a282 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -7,6 +7,9 @@ #define ZFP_VERSION_PATCH 0 /* library patch version number */ #define ZFP_VERSION_TWEAK 0 /* library tweak version number */ +/* defined for work in progress (indicates unofficial release) */ +#define ZFP_VERSION_DEVELOP 1 + /* codec version number (see also zfp_codec_version) */ #define ZFP_CODEC 5 From 6d7d2424ed082eb41d696036b26831636650a614 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 10 Aug 2022 13:35:59 -0700 Subject: [PATCH 144/277] Prevent LTO in libm dependency check (resolves #169) --- CMakeLists.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 2d0615aca..27d21f810 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -257,10 +257,10 @@ list(APPEND ppm_private_defs PPM_CHROMA=${PPM_CHROMA}) # Link libm only if necessary include(CheckCSourceCompiles) -check_c_source_compiles("#include\nfloat f; int main(){sqrt(f);return 0;}" HAVE_MATH) +check_c_source_compiles("#include\nint main(int n,char*v[]){return sqrt(n);}" HAVE_MATH) if(NOT HAVE_MATH) set(CMAKE_REQUIRED_LIBRARIES m) - check_c_source_compiles("#include\nfloat f; int main(){sqrt(f);return 0;}" HAVE_LIBM_MATH) + check_c_source_compiles("#include\nint main(int n,char*v[]){return sqrt(n);}" HAVE_LIBM_MATH) unset(CMAKE_REQUIRED_LIBRARIES) if(NOT HAVE_LIBM_MATH) message(FATAL_ERROR "Unable to use C math library functions (with or without -lm)") From c616bbec31fec8a4a3ba7c5c532ec19a0f2b2979 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 11 Aug 2022 10:58:12 -0700 Subject: [PATCH 145/277] Update CHANGELOG --- CHANGELOG.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3251d5155..ce19ef312 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,14 @@ Change Log --- +## Unreleased + +### Fixed + +- #169: `libm` dependency is not always correctly detected. + +--- + ## 1.0.0 (2022-08-01) This release is not ABI compatible with prior releases due to numerous changes From adfeb358435bb82350c2f91cd1b6d3edd1542cc1 Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Fri, 12 Aug 2022 17:01:01 -0400 Subject: [PATCH 146/277] feat(python): add full header reader for zfp streams --- python/zfpy.pxd | 7 ++++++- python/zfpy.pyx | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/python/zfpy.pxd b/python/zfpy.pxd index 60922c6b8..b5dc0e47e 100644 --- a/python/zfpy.pxd +++ b/python/zfpy.pxd @@ -31,7 +31,11 @@ cdef extern from "zfp.h": ptrdiff_t sx, sy, sz, sw void* data ctypedef struct zfp_stream: - pass + cython.uint minbits + cython.uint maxbits + cython.uint maxprec + int minexp + bitstream* stream ctypedef int zfp_bool @@ -53,6 +57,7 @@ cdef extern from "zfp.h": cython.uint zfp_stream_set_precision(zfp_stream* stream, cython.uint precision) double zfp_stream_set_accuracy(zfp_stream* stream, double tolerance) zfp_mode zfp_stream_set_mode(zfp_stream* stream, stdint.uint64_t mode) + zfp_mode zfp_stream_compression_mode(const zfp_stream* stream) zfp_field* zfp_field_alloc() zfp_field* zfp_field_1d(void* pointer, zfp_type, size_t nx) zfp_field* zfp_field_2d(void* pointer, zfp_type, size_t nx, size_t ny) diff --git a/python/zfpy.pyx b/python/zfpy.pyx index 12b25cb72..39b755ce0 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -351,3 +351,50 @@ cpdef np.ndarray decompress_numpy( stream_close(bstream) return output + +cpdef dict header(const uint8_t[::1] compressed_data): + """Return stream header information in a python dict.""" + if compressed_data is None: + raise TypeError("compressed_data cannot be None") + + cdef const void* comp_data_pointer = &compressed_data[0] + cdef zfp_field* field = zfp_field_alloc() + cdef bitstream* bstream = stream_open( + comp_data_pointer, + len(compressed_data) + ) + cdef zfp_stream* stream = zfp_stream_open(bstream) + cdef zfp_mode mode + + cdef zfp_stream strm + + try: + if zfp_read_header(stream, field, HEADER_FULL) == 0: + raise ValueError("Failed to read required zfp header") + + mode = zfp_stream_compression_mode(stream) + strm = stream[0] + + return { + "nx": int(field.nx), + "ny": int(field.ny), + "nz": int(field.nz), + "nw": int(field.nw), + "type": ztype_to_dtype(field._type), + "sx": int(field.sx), + "sy": int(field.sy), + "sz": int(field.sz), + "sw": int(field.sw), + "mode": mode, + "params": { + "minbits": strm.minbits, + "maxbits": strm.minbits, + "maxprec": strm.maxprec, + "minexp": strm.minexp, + }, + + } + finally: + zfp_field_free(field) + zfp_stream_close(stream) + stream_close(bstream) From fc0a19c54929b379a8f656447649377cbd9c30c2 Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Fri, 12 Aug 2022 17:04:55 -0400 Subject: [PATCH 147/277] fix: cast everything to python int --- python/zfpy.pyx | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/python/zfpy.pyx b/python/zfpy.pyx index 39b755ce0..a72b24629 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -385,14 +385,13 @@ cpdef dict header(const uint8_t[::1] compressed_data): "sy": int(field.sy), "sz": int(field.sz), "sw": int(field.sw), - "mode": mode, + "mode": int(mode), "params": { - "minbits": strm.minbits, - "maxbits": strm.minbits, - "maxprec": strm.maxprec, - "minexp": strm.minexp, + "minbits": int(strm.minbits), + "maxbits": int(strm.minbits), + "maxprec": int(strm.maxprec), + "minexp": int(strm.minexp), }, - } finally: zfp_field_free(field) From c05eba980fb5e6a97d657429769ed4cb4421b90f Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Sat, 13 Aug 2022 15:56:45 -0400 Subject: [PATCH 148/277] fix: remove strides from header dict --- python/zfpy.pyx | 4 ---- 1 file changed, 4 deletions(-) diff --git a/python/zfpy.pyx b/python/zfpy.pyx index a72b24629..db549456f 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -381,10 +381,6 @@ cpdef dict header(const uint8_t[::1] compressed_data): "nz": int(field.nz), "nw": int(field.nw), "type": ztype_to_dtype(field._type), - "sx": int(field.sx), - "sy": int(field.sy), - "sz": int(field.sz), - "sw": int(field.sw), "mode": int(mode), "params": { "minbits": int(strm.minbits), From 51afffc0520766ddfd6d1917291b424e73a723a1 Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Sat, 13 Aug 2022 16:30:18 -0400 Subject: [PATCH 149/277] fix: recommended changes --- python/zfpy.pxd | 13 ++++++------- python/zfpy.pyx | 44 ++++++++++++++++++++++++++++++++++++-------- 2 files changed, 42 insertions(+), 15 deletions(-) diff --git a/python/zfpy.pxd b/python/zfpy.pxd index b5dc0e47e..a2e73973a 100644 --- a/python/zfpy.pxd +++ b/python/zfpy.pxd @@ -31,11 +31,7 @@ cdef extern from "zfp.h": ptrdiff_t sx, sy, sz, sw void* data ctypedef struct zfp_stream: - cython.uint minbits - cython.uint maxbits - cython.uint maxprec - int minexp - bitstream* stream + pass ctypedef int zfp_bool @@ -57,7 +53,10 @@ cdef extern from "zfp.h": cython.uint zfp_stream_set_precision(zfp_stream* stream, cython.uint precision) double zfp_stream_set_accuracy(zfp_stream* stream, double tolerance) zfp_mode zfp_stream_set_mode(zfp_stream* stream, stdint.uint64_t mode) - zfp_mode zfp_stream_compression_mode(const zfp_stream* stream) + zfp_mode zfp_stream_compression_mode(zfp_stream* stream) + double zfp_stream_accuracy(zfp_stream* stream) + double zfp_stream_rate(zfp_stream* stream, cython.uint dims) + cython.uint zfp_stream_precision(const zfp_stream* stream) zfp_field* zfp_field_alloc() zfp_field* zfp_field_1d(void* pointer, zfp_type, size_t nx) zfp_field* zfp_field_2d(void* pointer, zfp_type, size_t nx, size_t ny) @@ -75,5 +74,5 @@ cdef extern from "zfp.h": size_t zfp_decompress(zfp_stream* stream, zfp_field* field) nogil size_t zfp_write_header(zfp_stream* stream, const zfp_field* field, cython.uint mask) size_t zfp_read_header(zfp_stream* stream, zfp_field* field, cython.uint mask) - + void zfp_stream_params(zfp_stream* stream, cython.uint* minbits, cython.uint* maxbits, cython.uint* maxprec, int* minexp); cdef gen_padded_int_list(orig_array, pad=*, length=*) diff --git a/python/zfpy.pyx b/python/zfpy.pyx index db549456f..bb72f4096 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -74,6 +74,20 @@ cpdef ztype_to_dtype(zfp_type ztype): except KeyError: raise ValueError("Unsupported zfp_type {}".format(ztype)) +zfp_mode_map = { + zfp_mode_null: "null", + zfp_mode_expert: "expert", + zfp_mode_reversible: "reversible", + zfp_mode_fixed_accuracy: "tolerance", + zfp_mode_fixed_precision: "precision", + zfp_mode_fixed_rate: "rate", +} +cpdef zmode_to_str(zfp_mode zmode): + try: + return zfp_mode_map[zmode] + except KeyError: + raise ValueError("Unsupported zfp_mode {}".format(zmode)) + cdef zfp_field* _init_field(np.ndarray arr): shape = arr.shape cdef int ndim = arr.ndim @@ -366,14 +380,22 @@ cpdef dict header(const uint8_t[::1] compressed_data): cdef zfp_stream* stream = zfp_stream_open(bstream) cdef zfp_mode mode - cdef zfp_stream strm + cdef unsigned int minbits = 0 + cdef unsigned int maxbits = 0 + cdef unsigned int maxprec = 0 + cdef int minexp = 0 try: if zfp_read_header(stream, field, HEADER_FULL) == 0: raise ValueError("Failed to read required zfp header") mode = zfp_stream_compression_mode(stream) - strm = stream[0] + + ndim = 0 + for dim in [field.nx, field.ny, field.nz, field.nw]: + ndim += int(dim > 0) + + zfp_stream_params(stream, &minbits, &maxbits, &maxprec, &minexp) return { "nx": int(field.nx), @@ -381,12 +403,18 @@ cpdef dict header(const uint8_t[::1] compressed_data): "nz": int(field.nz), "nw": int(field.nw), "type": ztype_to_dtype(field._type), - "mode": int(mode), - "params": { - "minbits": int(strm.minbits), - "maxbits": int(strm.minbits), - "maxprec": int(strm.maxprec), - "minexp": int(strm.minexp), + "mode": zmode_to_str(mode), + "config": { + "mode": int(mode), + "tolerance": float(zfp_stream_accuracy(stream)), + "rate": float(zfp_stream_rate(stream, ndim)), + "precision": int(zfp_stream_precision(stream)), + "expert": { + "minbits": int(minbits), + "maxbits": int(minbits), + "maxprec": int(maxprec), + "minexp": int(minexp), + }, }, } finally: From b2ad09e554b424387c5e1c588e33e0de610b0bd2 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 26 Aug 2022 10:49:30 -0700 Subject: [PATCH 150/277] Fix issue #176 --- CHANGELOG.md | 1 + zfp-config.cmake.in | 17 +++++++++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ce19ef312..e199771f3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ Change Log ### Fixed +- #176: `CFP` API is not exposed via CMake configuration file - #169: `libm` dependency is not always correctly detected. --- diff --git a/zfp-config.cmake.in b/zfp-config.cmake.in index 87ceb5d2c..c98096546 100644 --- a/zfp-config.cmake.in +++ b/zfp-config.cmake.in @@ -5,10 +5,17 @@ # ZFP_LIBRARIES - libraries to link against # ZFP_WITH_OPENMP - indicates if the zfp library has been built with OpenMP support # ZFP_WITH_CUDA - indicates if the zfp library has been built with CUDA support +# ZFP_CFP_ENABLED - indicated if the cfp library has been built # # And the following imported targets: # zfp::zfp # +# If cfp is enabled the following variabled are also defined +# CFP_INCLUDE_DIRS - include directories for cfp +# CFP_LIBRARIES - libraries to link against (cfp only) +# +# As well as the following imported targets: +# zfp::cfp include("${CMAKE_CURRENT_LIST_DIR}/zfp-config-version.cmake") @@ -18,10 +25,16 @@ find_package_handle_standard_args(${CMAKE_FIND_PACKAGE_NAME} CONFIG_MODE) if(NOT TARGET zfp::zfp) include("${CMAKE_CURRENT_LIST_DIR}/zfp-targets.cmake") + set(ZFP_LIBRARIES "zfp::zfp") + get_target_property(ZFP_INCLUDE_DIRS zfp::zfp INTERFACE_INCLUDE_DIRECTORIES) endif() -set(ZFP_LIBRARIES zfp::zfp) -get_target_property(ZFP_INCLUDE_DIRS zfp::zfp INTERFACE_INCLUDE_DIRECTORIES) +set(ZFP_CFP_ENABLED @BUILD_CFP@) +if(ZFP_CFP_ENABLED AND NOT TARGET zfp::cfp) + include("${CMAKE_CURRENT_LIST_DIR}/cfp-targets.cmake") + set(CFP_LIBRARIES "zfp::cfp") + get_target_property(CFP_INCLUDE_DIRS zfp::cfp INTERFACE_INCLUDE_DIRECTORIES) +endif() set(ZFP_WITH_OPENMP @ZFP_WITH_OPENMP@) if(ZFP_WITH_OPENMP) From 228c7a48adbf66d24263c025a874c5ef5dc3921e Mon Sep 17 00:00:00 2001 From: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com> Date: Mon, 12 Sep 2022 21:10:33 +0200 Subject: [PATCH 151/277] Update GitHub Actions --- .github/workflows/main.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index eeb42d06c..6939244fe 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -46,10 +46,10 @@ jobs: omp: ON target: all steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v3 - name: Setup Python - uses: actions/setup-python@v2 + uses: actions/setup-python@v4 with: python-version: '3.x' architecture: x64 @@ -60,7 +60,7 @@ jobs: python -m pip install numpy - name: Setup Python - uses: actions/setup-python@v2 + uses: actions/setup-python@v4 with: python-version: '3.x' architecture: x64 From 50dfaacdc3093a543b9b40b4558bb44e9e14a41b Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Tue, 27 Sep 2022 16:00:13 -0400 Subject: [PATCH 152/277] build: use oldest numpy ABI to ensure maximum compatibility --- .github/workflows/main.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index eeb42d06c..732a9e959 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -57,7 +57,7 @@ jobs: - name: Install zfpy dependencies run: | python -m pip install cython - python -m pip install numpy + python -m pip install oldest-supported-numpy - name: Setup Python uses: actions/setup-python@v2 @@ -68,7 +68,7 @@ jobs: - name: Install zfpy dependencies run: | python -m pip install cython - python -m pip install numpy + python -m pip install oldest-supported-numpy - name: Setup MSBuild (Windows) id: msbuild From 4de8c1b7ecce19265c91075b3088cfddf4a88a7f Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 27 Sep 2022 13:12:18 -0700 Subject: [PATCH 153/277] CMake RPath fix (issue #181) --- CMakeLists.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/CMakeLists.txt b/CMakeLists.txt index 27d21f810..eaaa8548d 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -51,6 +51,9 @@ if(NOT CMAKE_ARCHIVE_OUTPUT_DIRECTORY) set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${ZFP_BINARY_DIR}/${CMAKE_INSTALL_LIBDIR}) endif() +# Setup RPath +set(CMAKE_INSTALL_RPATH ${CMAKE_INSTALL_FULL_LIBDIR}) + #------------------------------------------------------------------------------# # Top level options #------------------------------------------------------------------------------# From 28cb77742ec7a3bf295b23a131fc0a6652f8d344 Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Fri, 12 Aug 2022 11:59:31 -0700 Subject: [PATCH 154/277] Fix zfpy type issues (issue #171) --- python/CMakeLists.txt | 2 +- python/scikit-build-cmake/FindCython.cmake | 11 +++++++++-- python/zfpy.pxd | 1 + tests/python/test_utils.pyx | 1 + 4 files changed, 12 insertions(+), 3 deletions(-) diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index 9c06d453c..f2c1041b9 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -6,7 +6,7 @@ include(FindNumPy) find_package(PythonInterp REQUIRED) find_package(PythonLibs REQUIRED) find_package(PythonExtensions REQUIRED) -find_package(Cython REQUIRED) +find_package(Cython 0.28 REQUIRED) # >= v0.28 required for const memoryview support find_package(NumPy REQUIRED) include_directories(${ZFP_SOURCE_DIR}/include) diff --git a/python/scikit-build-cmake/FindCython.cmake b/python/scikit-build-cmake/FindCython.cmake index 5f2ce6e09..73534e9c9 100644 --- a/python/scikit-build-cmake/FindCython.cmake +++ b/python/scikit-build-cmake/FindCython.cmake @@ -36,16 +36,23 @@ # Use the Cython executable that lives next to the Python executable # if it is a local installation. + find_package(PythonInterp) if(PYTHONINTERP_FOUND) - get_filename_component(_python_path ${PYTHON_EXECUTABLE} PATH) + execute_process(COMMAND ${PYTHON_EXECUTABLE} -c "from os import path; import cython; print(path.dirname(cython.__file__))" + OUTPUT_VARIABLE CYTHON_PATH + ERROR_VARIABLE CYTHON_PATH + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_STRIP_TRAILING_WHITESPACE) + find_program(CYTHON_EXECUTABLE NAMES cython cython.bat cython3 - HINTS ${_python_path} + HINTS ${CYTHON_PATH} DOC "path to the cython executable") else() find_program(CYTHON_EXECUTABLE NAMES cython cython.bat cython3 + HINTS ${CYTHON_PATH} DOC "path to the cython executable") endif() diff --git a/python/zfpy.pxd b/python/zfpy.pxd index 0d9a4ef3b..c92c29833 100644 --- a/python/zfpy.pxd +++ b/python/zfpy.pxd @@ -1,5 +1,6 @@ import cython cimport libc.stdint as stdint +from libc.stddef cimport ptrdiff_t cdef extern from "zfp/bitstream.h": cdef struct bitstream: diff --git a/tests/python/test_utils.pyx b/tests/python/test_utils.pyx index 2915b4a9b..6b486151b 100644 --- a/tests/python/test_utils.pyx +++ b/tests/python/test_utils.pyx @@ -2,6 +2,7 @@ import cython from libc.stdlib cimport malloc, free cimport libc.stdint as stdint +from libc.stddef cimport ptrdiff_t from cython cimport view from itertools import islice, repeat, chain From c578028e047a85f6c42d522cb1ea64b78aff4900 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 11 Oct 2022 13:33:39 -0700 Subject: [PATCH 155/277] Add BUILD_TESTING_FULL option (issue #177) --- CMakeLists.txt | 4 +- cmake/appveyor.cmake | 1 + docs/source/installation.rst | 9 +- tests/CMakeLists.txt | 288 ++++++++++++++++++----------------- tests/gitlab/gitlab-ci.yml | 4 +- 5 files changed, 158 insertions(+), 148 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index eaaa8548d..52ae1584e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -274,7 +274,7 @@ endif() # Add source code #------------------------------------------------------------------------------# include(CTest) -if(BUILD_TESTING) +if(BUILD_TESTING OR BUILD_TESTING_FULL) enable_testing() endif() @@ -318,7 +318,7 @@ if(BUILD_EXAMPLES) add_subdirectory(examples) endif() -if(BUILD_TESTING) +if(BUILD_TESTING OR BUILD_TESTING_FULL) # Disable gtest install to prevent clobbering existing installations option(INSTALL_GMOCK "Install Googlemock" OFF) option(INSTALL_GTEST "Install Googletest" OFF) diff --git a/cmake/appveyor.cmake b/cmake/appveyor.cmake index 1e803713e..29cc79069 100644 --- a/cmake/appveyor.cmake +++ b/cmake/appveyor.cmake @@ -12,6 +12,7 @@ set(CTEST_CMAKE_GENERATOR "${GENERATOR}") set(CTEST_BUILD_NAME "$ENV{APPVEYOR_REPO_BRANCH}-${job_details}") set(cfg_options -DCMAKE_BUILD_TYPE=$ENV{BUILD_TYPE} + -DBUILD_TESTING_FULL=ON -DBUILD_CFP=${BUILD_CFP} -DBUILD_ZFPY=${BUILD_ZFPY} -DZFP_WITH_OPENMP=${BUILD_OPENMP} diff --git a/docs/source/installation.rst b/docs/source/installation.rst index d43038ce8..fb11314b8 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -177,11 +177,16 @@ Regardless of the settings below, |libzfp| will always be built. .. c:macro:: BUILD_TESTING - Build |testzfp| and (when on the GitHub - `develop branch `__) unit tests. + Build |testzfp| tests. Default: on. +.. c:macro:: BUILD_TESTING_FULL + + Build all unit tests. + Default: off. + + .. c:macro:: BUILD_SHARED_LIBS Build shared objects (:file:`.so`, :file:`.dylib`, or :file:`.dll` files). diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 7d0ded622..ac47757e7 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -1,154 +1,158 @@ -set(CMAKE_CXX_STANDARD 11) - -# CMAKE_SH-NOTFOUND needed for mingw builds -if(MINGW) - list(APPEND CMOCKA_ARGS "-DCMAKE_SH=CMAKE_SH-NOTFOUND") - list(APPEND GTEST_ARGS "-DCMAKE_SH=CMAKE_SH-NOTFOUND") -endif() - -# clone cmocka 1.1.0 into /build -list(APPEND CMOCKA_ARGS "-DWITH_STATIC_LIB=ON;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") - -include(ExternalProject) -ExternalProject_Add( - cmocka_cloned - GIT_REPOSITORY https://gitlab.com/cmocka/cmocka.git - GIT_TAG cmocka-1.1.0 - SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" - BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" - CMAKE_ARGS "${CMOCKA_ARGS}" - INSTALL_COMMAND "" - STEP_TARGETS build - EXCLUDE_FROM_ALL TRUE -) -ExternalProject_Get_Property(cmocka_cloned source_dir binary_dir) - -# name static library cmocka, wire up against cmocka_cloned -add_library(cmocka STATIC IMPORTED GLOBAL) - -# choose proper library path & extension -if(MSVC) - set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka.lib") -else() - set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka.a") -endif() -set_property(TARGET cmocka - PROPERTY - IMPORTED_LOCATION "${IMPORTED_LOCATION_PATH}" -) - -add_dependencies(cmocka cmocka_cloned) -include_directories(${source_dir}/include) - -# include home dir so #include statements are clear in test files -include_directories(${ZFP_SOURCE_DIR} ${ZFP_SOURCE_DIR}/include) -# access to constants/ and utils/ -include_directories(${CMAKE_CURRENT_SOURCE_DIR}) - -# suppress warnings for all targets -if(CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_C_COMPILER_ID STREQUAL "Clang") - add_compile_options(-Wno-unused-function) -endif() -# -Wno-variadic-macros was not working for gcc...revisit -if(CMAKE_C_COMPILER_ID STREQUAL "Clang") - add_compile_options(-Wno-gnu-zero-variadic-macro-arguments) -endif() -# suppress googletest warning "conversion from 'float' to 'testing::internal::BiggestInt', possible loss of data" -if(MSVC) - add_compile_options(/wd4244) -endif() - - -add_subdirectory(utils) -add_subdirectory(src) - -if(BUILD_CFP) - add_subdirectory(cfp) -endif() - -if(BUILD_ZFORP) - add_subdirectory(fortran) -endif() - -# needed to compile gtest on MSVC -if(MSVC) - list(APPEND GTEST_ARGS "/D:_SILENCE_TR1_DEPRECATION_NAMESPACE_WARNING=1") +if(BUILD_TESTING OR BUILD_TESTING_FULL) + # testzfp + add_executable(testzfp testzfp.cpp) + target_link_libraries(testzfp zfp) + target_compile_definitions(testzfp PRIVATE ${zfp_compressed_array_defs}) + add_test(NAME testzfp COMMAND testzfp) + + # testviews + add_executable(testviews testviews.cpp) + if(ZFP_WITH_OPENMP) + target_link_libraries(testviews zfp OpenMP::OpenMP_C) + else() + target_link_libraries(testviews zfp) + endif() + target_compile_definitions(testviews PRIVATE ${zfp_compressed_array_defs}) + add_test(NAME testviews COMMAND testviews) endif() -# TODO: spend time getting googletest to compile on MinGW -# checksums are generated through C tests, no need to compile C++ tests -if((NOT MINGW) AND (NOT DEFINED ZFP_OMP_TESTS_ONLY) AND (NOT PRINT_CHECKSUMS)) - # clone googletest into build/ - configure_file(CMakeLists.txt.in ${ZFP_BINARY_DIR}/tests/googletest-download/CMakeLists.txt) - execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" ${GTEST_ARGS} . - RESULT_VARIABLE result - WORKING_DIRECTORY ${ZFP_BINARY_DIR}/tests/googletest-download - ) - - if(result) - message(FATAL_ERROR "CMake step for googletest failed: ${result}") +if(BUILD_TESTING_FULL) + set(CMAKE_CXX_STANDARD 11) + + # CMAKE_SH-NOTFOUND needed for mingw builds + if(MINGW) + list(APPEND CMOCKA_ARGS "-DCMAKE_SH=CMAKE_SH-NOTFOUND") + list(APPEND GTEST_ARGS "-DCMAKE_SH=CMAKE_SH-NOTFOUND") endif() - # build gtest - execute_process(COMMAND ${CMAKE_COMMAND} --build . - RESULT_VARIABLE result - WORKING_DIRECTORY ${ZFP_BINARY_DIR}/tests/googletest-download + + # clone cmocka 1.1.0 into /build + list(APPEND CMOCKA_ARGS "-DWITH_STATIC_LIB=ON;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") + + include(ExternalProject) + ExternalProject_Add( + cmocka_cloned + GIT_REPOSITORY https://gitlab.com/cmocka/cmocka.git + GIT_TAG cmocka-1.1.0 + SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" + BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" + CMAKE_ARGS "${CMOCKA_ARGS}" + INSTALL_COMMAND "" + STEP_TARGETS build + EXCLUDE_FROM_ALL TRUE ) - if(result) - message(FATAL_ERROR "Build step for googletest failed: ${result}") + ExternalProject_Get_Property(cmocka_cloned source_dir binary_dir) + + # name static library cmocka, wire up against cmocka_cloned + add_library(cmocka STATIC IMPORTED GLOBAL) + + # choose proper library path & extension + if(MSVC) + set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka.lib") + else() + set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka.a") endif() - - set(gtest_force_shared_crt ON CACHE BOOL "" FORCE) - - add_subdirectory(${ZFP_BINARY_DIR}/tests/googletest-src - ${ZFP_BINARY_DIR}/tests/googletest-build + set_property(TARGET cmocka + PROPERTY + IMPORTED_LOCATION "${IMPORTED_LOCATION_PATH}" ) - - if(CMAKE_VERSION VERSION_LESS 2.8.11) - include_directories("${gtest_SOURCE_DIR}/include") + + add_dependencies(cmocka cmocka_cloned) + include_directories(${source_dir}/include) + + # include home dir so #include statements are clear in test files + include_directories(${ZFP_SOURCE_DIR} ${ZFP_SOURCE_DIR}/include) + # access to constants/ and utils/ + include_directories(${CMAKE_CURRENT_SOURCE_DIR}) + + # suppress warnings for all targets + if(CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_C_COMPILER_ID STREQUAL "Clang") + add_compile_options(-Wno-unused-function) endif() - - # needed to compile zfp tests with gtest on MSVC + # -Wno-variadic-macros was not working for gcc...revisit + if(CMAKE_C_COMPILER_ID STREQUAL "Clang") + add_compile_options(-Wno-gnu-zero-variadic-macro-arguments) + endif() + # suppress googletest warning "conversion from 'float' to 'testing::internal::BiggestInt', possible loss of data" if(MSVC) - target_compile_definitions(gtest_main INTERFACE GTEST_LINKED_AS_SHARED_LIBRARY=1) + add_compile_options(/wd4244) endif() - - add_subdirectory(array) -endif() - - -# testzfp -add_executable(testzfp testzfp.cpp) -target_link_libraries(testzfp zfp) -target_compile_definitions(testzfp PRIVATE ${zfp_compressed_array_defs}) - -option(ZFP_BUILD_TESTING_SMALL "Enable small-sized array testing" ON) -if(ZFP_BUILD_TESTING_SMALL) - foreach(D IN ITEMS 1 2 3 4) - foreach(P IN ITEMS 32 64) - add_test(NAME small-arrays-${D}d-fp${P} COMMAND testzfp small ${D}d fp${P}) + + + add_subdirectory(utils) + add_subdirectory(src) + + if(BUILD_CFP) + add_subdirectory(cfp) + endif() + + if(BUILD_ZFORP) + add_subdirectory(fortran) + endif() + + # needed to compile gtest on MSVC + if(MSVC) + list(APPEND GTEST_ARGS "/D:_SILENCE_TR1_DEPRECATION_NAMESPACE_WARNING=1") + endif() + + # TODO: spend time getting googletest to compile on MinGW + # checksums are generated through C tests, no need to compile C++ tests + if((NOT MINGW) AND (NOT DEFINED ZFP_OMP_TESTS_ONLY) AND (NOT PRINT_CHECKSUMS)) + # clone googletest into build/ + configure_file(CMakeLists.txt.in ${ZFP_BINARY_DIR}/tests/googletest-download/CMakeLists.txt) + execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" ${GTEST_ARGS} . + RESULT_VARIABLE result + WORKING_DIRECTORY ${ZFP_BINARY_DIR}/tests/googletest-download + ) + + if(result) + message(FATAL_ERROR "CMake step for googletest failed: ${result}") + endif() + # build gtest + execute_process(COMMAND ${CMAKE_COMMAND} --build . + RESULT_VARIABLE result + WORKING_DIRECTORY ${ZFP_BINARY_DIR}/tests/googletest-download + ) + if(result) + message(FATAL_ERROR "Build step for googletest failed: ${result}") + endif() + + set(gtest_force_shared_crt ON CACHE BOOL "" FORCE) + + add_subdirectory(${ZFP_BINARY_DIR}/tests/googletest-src + ${ZFP_BINARY_DIR}/tests/googletest-build + ) + + if(CMAKE_VERSION VERSION_LESS 2.8.11) + include_directories("${gtest_SOURCE_DIR}/include") + endif() + + # needed to compile zfp tests with gtest on MSVC + if(MSVC) + target_compile_definitions(gtest_main INTERFACE GTEST_LINKED_AS_SHARED_LIBRARY=1) + endif() + + add_subdirectory(array) + endif() + + option(ZFP_BUILD_TESTING_SMALL "Enable small-sized array testing" ON) + if(ZFP_BUILD_TESTING_SMALL) + foreach(D IN ITEMS 1 2 3 4) + foreach(P IN ITEMS 32 64) + add_test(NAME small-arrays-${D}d-fp${P} COMMAND testzfp small ${D}d fp${P}) + endforeach() endforeach() - endforeach() -endif() - -option(ZFP_BUILD_TESTING_LARGE "Enable large-sized array testing" OFF) -if(ZFP_BUILD_TESTING_LARGE) - foreach(D IN ITEMS 1 2 3 4) - foreach(P IN ITEMS 32 64) - add_test(NAME large-arrays-${D}d-fp${P} COMMAND testzfp large ${D}d fp${P}) + endif() + + option(ZFP_BUILD_TESTING_LARGE "Enable large-sized array testing" OFF) + if(ZFP_BUILD_TESTING_LARGE) + foreach(D IN ITEMS 1 2 3 4) + foreach(P IN ITEMS 32 64) + add_test(NAME large-arrays-${D}d-fp${P} COMMAND testzfp large ${D}d fp${P}) + endforeach() endforeach() - endforeach() -endif() - -# testviews -add_executable(testviews testviews.cpp) -if(ZFP_WITH_OPENMP) - target_link_libraries(testviews zfp OpenMP::OpenMP_C) -else() - target_link_libraries(testviews zfp) -endif() -target_compile_definitions(testviews PRIVATE ${zfp_compressed_array_defs}) -add_test(NAME testviews COMMAND testviews) - -if(BUILD_ZFPY) - add_subdirectory(python) + endif() + + if(BUILD_ZFPY) + add_subdirectory(python) + endif() endif() diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 5a7ee9bf5..794dc8390 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -50,7 +50,7 @@ stages: script: - mkdir build - cd build - - cmake -DBUILD_TESTING=ON -DBUILD_UTILITIES=OFF -DZFP_WITH_CUDA=OFF ${ci_cmake_flags} .. + - cmake -DBUILD_TESTING_FULL=ON -DBUILD_UTILITIES=OFF -DZFP_WITH_CUDA=OFF ${ci_cmake_flags} .. - make -j extends: [.build] @@ -63,7 +63,7 @@ stages: script: - mkdir build - cd build - - cmake -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=OFF -DBUILD_UTILITIES=OFF ${ci_cmake_flags} .. + - cmake -DBUILD_TESTING_FULL=ON -DZFP_WITH_OPENMP=OFF -DBUILD_UTILITIES=OFF ${ci_cmake_flags} .. - make -j extends: [.build] From b50a4613e7fea08d09e993d844cb22b3255f6650 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 11 Oct 2022 13:40:15 -0700 Subject: [PATCH 156/277] update github actions flag --- .github/workflows/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 59ede46c3..9d9406d56 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -91,7 +91,7 @@ jobs: - name: Run CMake id: cmake - run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING=ON -DZFP_WITH_OPENMP=${{matrix.omp}} -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") + run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING_FULL=ON -DZFP_WITH_OPENMP=${{matrix.omp}} -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") - name: Build id: build From 54c8706dee120b22709fe00a17e9c8187551c0b8 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 13 Oct 2022 11:50:59 -0700 Subject: [PATCH 157/277] Add BUILD_TESTING_FULL to documentation [skip ci] --- CHANGELOG.md | 1 + README.md | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e199771f3..7387be1a9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ Change Log ### Fixed +- # 177: Full test suite not in release - #176: `CFP` API is not exposed via CMake configuration file - #169: `libm` dependency is not always correctly detected. diff --git a/README.md b/README.md index 8f207f162..8698f024a 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,8 @@ zfp may be built using either [CMake](https://cmake.org/) or This builds the zfp library in the `build/lib` directory and the zfp command-line executable in the `build/bin` directory. It then runs -the regression tests. +the regression tests. The full test suite may be run by enabling the +`BUILD_TESTING_FULL` CMake option during the build step. zfp may also be built using GNU make: From ff3a339264f59e3409c8534ab20fe2ac126fb4e7 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 8 Dec 2022 10:30:13 -0800 Subject: [PATCH 158/277] update gitlabCI user --- tests/gitlab/gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 794dc8390..cd97ee486 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -5,6 +5,8 @@ variables: GIT_SUBMODULE_STRATEGY: recursive LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc -t 00:15:00" + CUSTOM_CI_BUILDS_DIR: "/usr/workspace/zfp/gitlab-ci" + LLNL_SERVICE_USER: zfp stages: - build From 8a1dd0e0e63711921f2694747601f8971c740fef Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 8 Dec 2022 16:08:10 -0800 Subject: [PATCH 159/277] Revert gitlab change --- tests/gitlab/gitlab-ci.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index cd97ee486..794dc8390 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -5,8 +5,6 @@ variables: GIT_SUBMODULE_STRATEGY: recursive LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc -t 00:15:00" - CUSTOM_CI_BUILDS_DIR: "/usr/workspace/zfp/gitlab-ci" - LLNL_SERVICE_USER: zfp stages: - build From 0d7de48d7029328818cfb1af4d99b02e87d183d8 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Thu, 26 Jan 2023 11:39:35 -0800 Subject: [PATCH 160/277] Create actions debug workflow for macos --- .github/workflows/debug_macos.yml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 .github/workflows/debug_macos.yml diff --git a/.github/workflows/debug_macos.yml b/.github/workflows/debug_macos.yml new file mode 100644 index 000000000..db5e9d344 --- /dev/null +++ b/.github/workflows/debug_macos.yml @@ -0,0 +1,26 @@ +name: Debug (MacOS) +on: [workflow_dispatch] +jobs: + debug: + runs-on: macos-latest + steps: + - name: Checkout Zfp + uses: actions/checkout@v3 + + - name: Setup Python + uses: actions/setup-python@v4 + with: + python-version: '3.x' + architecture: x64 + + - name: Install Zfpy Dependencies + run: | + python -m pip install cython + python -m pip install oldest-supported-numpy + + - name: Install OpenMP + run: | + brew install libomp + + - name: Setup Tmate Session + uses: mxschmitt/action-tmate@v3 From c4f175667568b70462de8d806a63773b3eb1fff7 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Thu, 26 Jan 2023 13:15:33 -0800 Subject: [PATCH 161/277] Create actions debug workflow for linux [skip ci] --- .github/workflows/debug-linux.yml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 .github/workflows/debug-linux.yml diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml new file mode 100644 index 000000000..94e364092 --- /dev/null +++ b/.github/workflows/debug-linux.yml @@ -0,0 +1,26 @@ +name: Debug (Linux) +on: [workflow_dispatch] +jobs: + debug: + runs-on: ubuntu-latest + steps: + - name: Checkout Zfp + uses: actions/checkout@v3 + + - name: Setup Python + uses: actions/setup-python@v4 + with: + python-version: '3.x' + architecture: x64 + + - name: Install Zfpy Dependencies + run: | + python -m pip install cython + python -m pip install oldest-supported-numpy + + - name: Install OpenMP + run: | + sudo apt-get update; sudo apt-get install -y libomp5 libomp-dev + + - name: Setup Tmate Session + uses: mxschmitt/action-tmate@v3 From ee10feb737f1928acbb8b9bebd350f511b1692b9 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Tue, 31 Jan 2023 13:36:12 -0800 Subject: [PATCH 162/277] squash merge misc/actions-fixes into develop --- .github/workflows/main.yml | 71 ++++++++++-------------------------- tests/CMakeLists.txt | 6 +-- tests/CMakeLists.txt.in | 2 +- tests/gitlab/gitlab-ci.yml | 21 ++++++++++- tests/gitlab/pascal-jobs.yml | 1 + tests/gitlab/quartz-jobs.yml | 21 ++--------- 6 files changed, 47 insertions(+), 75 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 9d9406d56..61d0f0c57 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -21,30 +21,18 @@ jobs: omp: ON target: all - #- os: windows-latest - # cxx_compiler: msbuild - # c_compiler: msbuild - # omp: ON - # target: ALL_BUILD - - #- os: windows-latest - # cxx_compiler: x86_64-w64-mingw32-g++ - # c_compiler: x86_64-w64-mingw32-gcc - # omp: ON - # target: all - # generator: '-G "MinGW Makefiles"' - + - os: macos-latest + cxx_compiler: g++-11 + c_compiler: gcc-11 + omp: ON + target: all + - os: macos-latest cxx_compiler: clang++ c_compiler: clang omp: OFF target: all - - - os: macos-latest - cxx_compiler: g++-10 - c_compiler: gcc-10 - omp: ON - target: all + steps: - uses: actions/checkout@v3 @@ -59,36 +47,19 @@ jobs: python -m pip install cython python -m pip install oldest-supported-numpy - - name: Setup Python - uses: actions/setup-python@v4 - with: - python-version: '3.x' - architecture: x64 - - - name: Install zfpy dependencies - run: | - python -m pip install cython - python -m pip install oldest-supported-numpy - - - name: Setup MSBuild (Windows) - id: msbuild - if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'msbuild'}} - uses: microsoft/setup-msbuild@v1.0.3 - - #- name: Setup MinGW (Windows) - # id: mingw - # if: ${{matrix.os == 'windows-latest' && matrix.cxx_compiler == 'x86_64-w64-mingw32-g++'}} - # uses: egor-tensin/setup-mingw@v2 - - - name: CI Settings - id: settings + - name: Setup OpenMP (Linux) + if: ${{matrix.os == 'ubuntu-latest' && matrix.cxx_compiler == 'clang++'}} + run: sudo apt-get update; sudo apt-get install -y libomp5 libomp-dev + + - name: Setup OpenMP (MacOS) + if: ${{matrix.os == 'macos-latest'}} run: | - echo "os: ${{matrix.os}}" - echo "compilers:" - echo " cxx: ${{matrix.cxx_compiler}}" - echo " c: ${{matrix.c_compiler}}" - echo "OpenMP: ${{matrix.omp}}" - + brew install libomp + echo "CC=$(brew --prefix llvm)/bin/clang" >> $GITHUB_ENV + echo "CXX=$(brew --prefix llvm)/bin/clang++" >> $GITHUB_ENV + echo "LDFLAGS=\"-L$(brew --prefix llvm)/lib\"" >> $GITHUB_ENV + echo "CPPFLAGS=\"-I$(brew --prefix llvm)/include\"" >> $GITHUB_ENV + - name: Run CMake id: cmake run: cmake -B ${{github.workspace}}/build ${{matrix.generator}} -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_CXX_COMPILER=${{matrix.cxx_compiler}} -DCMAKE_C_COMPILER=${{matrix.c_compiler}} -DBUILD_TESTING_FULL=ON -DZFP_WITH_OPENMP=${{matrix.omp}} -DBUILD_ZFPY=ON -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") @@ -101,7 +72,3 @@ jobs: id: test working-directory: ${{github.workspace}}/build run: ctest -C ${{env.BUILD_TYPE}} -VV - - # Interactive Debug -> see: https://github.com/mxschmitt/action-tmate - #- name: Setup Debug Session - # uses: mxschmitt/action-tmate@v3 diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index ac47757e7..4134a2734 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -32,7 +32,7 @@ if(BUILD_TESTING_FULL) ExternalProject_Add( cmocka_cloned GIT_REPOSITORY https://gitlab.com/cmocka/cmocka.git - GIT_TAG cmocka-1.1.0 + GIT_TAG cmocka-1.1.5 SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" CMAKE_ARGS "${CMOCKA_ARGS}" @@ -47,9 +47,9 @@ if(BUILD_TESTING_FULL) # choose proper library path & extension if(MSVC) - set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka.lib") + set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka-static.lib") else() - set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka.a") + set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka-static.a") endif() set_property(TARGET cmocka PROPERTY diff --git a/tests/CMakeLists.txt.in b/tests/CMakeLists.txt.in index efe6c3f6d..46e7b0880 100644 --- a/tests/CMakeLists.txt.in +++ b/tests/CMakeLists.txt.in @@ -6,7 +6,7 @@ include(ExternalProject) ExternalProject_Add( googletest GIT_REPOSITORY https://github.com/google/googletest.git - GIT_TAG 703bd9caab50b139428cea1aaff9974ebee5742e + GIT_TAG e2239ee6043f73722e7aa812a459f54a28552929 #703bd9caab50b139428cea1aaff9974ebee5742e SOURCE_DIR "${ZFP_BINARY_DIR}/tests/googletest-src" BINARY_DIR "${ZFP_BINARY_DIR}/tests/googletest-build" CONFIGURE_COMMAND "" diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 794dc8390..86fbf0e5f 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -26,6 +26,15 @@ stages: .build_cpu: before_script: + - |- + if [ "$ci_c_cmp" != "gcc" ]; then + module --latest load gcc + if (( $(gcc -dumpversion | sed 's/\..*//') < 5 )); then + echo "unable to find new enough gcc to support ${ci_c_cmp} build" + exit 1 + fi + export GXX_PATH=$(dirname $(which gcc))/../ + fi - module reset - module load $ci_cmake - module load $ci_cmp_mod @@ -50,8 +59,15 @@ stages: script: - mkdir build - cd build - - cmake -DBUILD_TESTING_FULL=ON -DBUILD_UTILITIES=OFF -DZFP_WITH_CUDA=OFF ${ci_cmake_flags} .. - - make -j + - |- + export ci_cmake_cmp_flags="" + if [ "$ci_c_cmp" == "icc" ]; then + export ci_cmake_cmp_flags="-DCMAKE_CXX_FLAGS=-gcc-name=${GXX_PATH}/bin/gcc -DCMAKE_C_FLAGS=-gcc-name=${GXX_PATH}/bin/gcc" + elif [ "$ci_c_cmp" == "clang" ]; then + export ci_cmake_cmp_flags="-DCMAKE_CXX_FLAGS=--gcc-toolchain=${GXX_PATH} -DCMAKE_C_FLAGS=--gcc-toolchain=${GXX_PATH}" + fi + - cmake -DBUILD_TESTING_FULL=ON -DBUILD_UTILITIES=OFF -DZFP_WITH_CUDA=OFF ${ci_cmake_flags} ${ci_cmake_cmp_flags} .. + - cmake --build . extends: [.build] .build_gpu: @@ -60,6 +76,7 @@ stages: - module load opt - module load $ci_cmake - module load $ci_cmp_mod + - module load $ci_gcc_mod script: - mkdir build - cd build diff --git a/tests/gitlab/pascal-jobs.yml b/tests/gitlab/pascal-jobs.yml index 3f3636558..1e5e5bead 100644 --- a/tests/gitlab/pascal-jobs.yml +++ b/tests/gitlab/pascal-jobs.yml @@ -6,6 +6,7 @@ cuda-10.1.168_build: variables: ci_cmake: "cmake/3.9.2" ci_cmp_mod: "cuda/10.1.168" + ci_gcc_mod: "gcc/8.3.1" extends: [.cuda, .pascal_build_gpu] needs: [] diff --git a/tests/gitlab/quartz-jobs.yml b/tests/gitlab/quartz-jobs.yml index 071728b32..80591ba66 100644 --- a/tests/gitlab/quartz-jobs.yml +++ b/tests/gitlab/quartz-jobs.yml @@ -30,33 +30,20 @@ cpp_clang-10.0.0_test: needs: [cpp_clang-10.0.0_build] -cpp_intel-19.0.4_build: +cpp_intel-2022.3_build: variables: ci_cmake: "cmake/3.9.2" ci_cxx_cmp: "icpc" ci_c_cmp: "icc" - ci_cmp_mod: "intel/19.0.4" + ci_cmp_mod: "intel/2022.3" extends: [.cpp, .quartz_build_cpu] needs: [] -cpp_intel-19.0.4_test: +cpp_intel-2022.3_test: extends: [.quartz_test_cpu] - needs: [cpp_intel-19.0.4_build] + needs: [cpp_intel-2022.3_build] -cpp_pgi-21.1_build: - variables: - ci_cmake: "cmake/3.9.2" - ci_cxx_cmp: "pgc++" - ci_c_cmp: "pgcc" - ci_cmp_mod: "pgi/21.1" - extends: [.cpp, .quartz_build_cpu] - needs: [] - -cpp_pgi-21.1_test: - extends: [.quartz_test_cpu] - needs: [cpp_pgi-21.1_build] - ######### # C CPU # From 2c7cdeaa211f97455da05c2e7c1b905cff402297 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Fri, 10 Feb 2023 15:32:00 -0800 Subject: [PATCH 163/277] Add code coverage action --- .github/workflows/coverage.yml | 49 ++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 .github/workflows/coverage.yml diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml new file mode 100644 index 000000000..0aac9702c --- /dev/null +++ b/.github/workflows/coverage.yml @@ -0,0 +1,49 @@ +on: + workflow_run: + workflows: [build] + types: [completed] + +jobs: + coverage: + runs-on: ubuntu-latest + if: ${{ github.event.workflow_run.conclusion == 'success' }} + steps: + - name: Checkout + uses: actions/checkout@v3 + + - name: Setup Python + uses: actions/setup-python@v4 + with: + python-version: '3.x' + architecture: x64 + + - name: Install Dependencies + run: |- + sudo apt install lcov + python -m pip install lcov_cobertura + + - name: Run CMake + run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-11 -DCMAKE_C_COMPILER=gcc-11 -DCMAKE_CXX_FLAGS="-fprofile-arcs -ftest-coverage" -DCMAKE_C_FLAGS="-fprofile-arcs -ftest-coverage" -DBUILD_TESTING_FULL=ON -DBUILD_CFP=ON -DZFP_WITH_OPENMP=ON + + - name: Build + run: cmake --build ${{github.workspace}}/build + + - name: Run Tests + working-directory: ${{github.workspace}}/build + run: ctest -j 8 + + - name: Generate Coverage Report + working-directory: ${{github.workspace}}/build + run: |- + lcov -c --directory ${{github.workspace}}/build --output-file coverage.info + lcov --remove coverage.info '${{github.workspace}}/build/tests/*' --remove coverage.info '${{github.workspace}}/tests/*' --remove coverage.info '/usr/include/*' -o coverage.info + lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml + + - name: Upload Report to Codecov + uses: codecov/codecov-action@v3 + with: + files: ${{github.workspace}}/build/coverage.xml + env_vars: Actions + fail_ci_if_error: true + verbose: true + dry_run: true From b78055fa32f265b5809a079bfe7a0807b9cf2023 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 10 Feb 2023 16:18:47 -0800 Subject: [PATCH 164/277] Organize actions and enable codecov upload --- .github/workflows/coverage.yml | 4 ++-- .github/workflows/debug-linux.yml | 2 ++ .github/workflows/{debug_macos.yml => debug-macos.yml} | 2 ++ .github/workflows/{main.yml => tests.yml} | 5 ++++- 4 files changed, 10 insertions(+), 3 deletions(-) rename .github/workflows/{debug_macos.yml => debug-macos.yml} (99%) rename .github/workflows/{main.yml => tests.yml} (99%) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 0aac9702c..1be5f3bf4 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -1,3 +1,5 @@ +name: Generate Coverage Report + on: workflow_run: workflows: [build] @@ -45,5 +47,3 @@ jobs: files: ${{github.workspace}}/build/coverage.xml env_vars: Actions fail_ci_if_error: true - verbose: true - dry_run: true diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 94e364092..1c6a36b85 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -1,5 +1,7 @@ name: Debug (Linux) + on: [workflow_dispatch] + jobs: debug: runs-on: ubuntu-latest diff --git a/.github/workflows/debug_macos.yml b/.github/workflows/debug-macos.yml similarity index 99% rename from .github/workflows/debug_macos.yml rename to .github/workflows/debug-macos.yml index db5e9d344..9170c521e 100644 --- a/.github/workflows/debug_macos.yml +++ b/.github/workflows/debug-macos.yml @@ -1,5 +1,7 @@ name: Debug (MacOS) + on: [workflow_dispatch] + jobs: debug: runs-on: macos-latest diff --git a/.github/workflows/main.yml b/.github/workflows/tests.yml similarity index 99% rename from .github/workflows/main.yml rename to .github/workflows/tests.yml index 61d0f0c57..50dcf5a99 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/tests.yml @@ -1,7 +1,10 @@ -name: build +name: Run Tests + on: push + env: BUILD_TYPE: Release + jobs: build: runs-on: ${{matrix.os}} From f15d00720e454cd70206a85d7013d8240ad38585 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Fri, 10 Feb 2023 16:44:50 -0800 Subject: [PATCH 165/277] Update badges and actions names --- .github/workflows/coverage.yml | 4 ++-- .github/workflows/tests.yml | 2 +- README.md | 5 +++-- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 1be5f3bf4..6f040669e 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -1,8 +1,8 @@ -name: Generate Coverage Report +name: Coverage Report on: workflow_run: - workflows: [build] + workflows: [Tests] types: [completed] jobs: diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 50dcf5a99..07c06974c 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -1,4 +1,4 @@ -name: Run Tests +name: Tests on: push diff --git a/README.md b/README.md index 8698f024a..1e25b75a4 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,9 @@ ZFP === -[![Github Actions Build Status](https://github.com/LLNL/zfp/actions/workflows/main.yml/badge.svg?branch=release1.0.0)](https://github.com/LLNL/zfp/actions/workflows/main.yml) -[![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/release1.0.0?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) +[![Github Actions Build Status](https://github.com/LLNL/zfp/workflows/Run%20Tests/badge.svg)](https://github.com/LLNL/zfp/actions/workflows/tests.yml) +[![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) [![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.0)](https://zfp.readthedocs.io/en/release1.0.0/) +[![codecov](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg?token=jqvMVvgRQ9)](https://codecov.io/gh/LLNL/zfp) zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high From fbca454079a465828129455885bae9d78ada8bcd Mon Sep 17 00:00:00 2001 From: diegojv Date: Tue, 11 Apr 2023 10:20:59 +0200 Subject: [PATCH 166/277] Modified the set_stride_xd functions in zfp.f90 so that stride values are actually passed by value and not by reference as they were. This resulted in segmentation faults when using strided compression with the Fortran bindings --- fortran/zfp.f90 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index f671d1449..8418b0e51 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -464,25 +464,25 @@ subroutine zfp_field_set_size_4d(field, nx, ny, nz, nw) bind(c, name="zfp_field_ subroutine zfp_field_set_stride_1d(field, sx) bind(c, name="zfp_field_set_stride_1d") import type(c_ptr), value :: field - integer(c_ptrdiff_t) :: sx + integer(c_ptrdiff_t), value :: sx end subroutine subroutine zfp_field_set_stride_2d(field, sx, sy) bind(c, name="zfp_field_set_stride_2d") import type(c_ptr), value :: field - integer(c_ptrdiff_t) :: sx, sy + integer(c_ptrdiff_t), value :: sx, sy end subroutine subroutine zfp_field_set_stride_3d(field, sx, sy, sz) bind(c, name="zfp_field_set_stride_3d") import type(c_ptr), value :: field - integer(c_ptrdiff_t) :: sx, sy, sz + integer(c_ptrdiff_t), value :: sx, sy, sz end subroutine subroutine zfp_field_set_stride_4d(field, sx, sy, sz, sw) bind(c, name="zfp_field_set_stride_4d") import type(c_ptr), value :: field - integer(c_ptrdiff_t) :: sx, sy, sz, sw + integer(c_ptrdiff_t), value :: sx, sy, sz, sw end subroutine function zfp_field_set_metadata(field, encoded_metadata) result(is_success) bind(c, name="zfp_field_set_metadata") From 3a90823d9232ea15b8beab9e55c7f81fa0cb6cf6 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 29 Apr 2023 12:38:41 -0700 Subject: [PATCH 167/277] Clarify that thread safety requires OpenMP [skip ci] --- docs/source/faq.rst | 2 +- docs/source/views.inc | 7 +++++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 8d6377bc2..77e20af2e 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -1036,7 +1036,7 @@ Q24: *Are zfp's compressed arrays and other data structures thread-safe?* A: Yes, compressed arrays can be made thread-safe; no, data structures like :c:type:`zfp_stream` and :c:type:`bitstream` are not necessarily thread-safe. As of |zfp| |viewsrelease|, thread-safe read and write access -to compressed arrays is provided through the use of +to compressed arrays via OpenMP threads is provided through the use of :ref:`private views `, although these come with certain restrictions and requirements such as the need for the user to enforce cache coherence. Please see the documentation on diff --git a/docs/source/views.inc b/docs/source/views.inc index 5c1364e09..4a14df297 100644 --- a/docs/source/views.inc +++ b/docs/source/views.inc @@ -654,6 +654,13 @@ that it maintains its own private cache rather than sharing the cache owned by the array. Multiple threads may thus access the same array in parallel through their own private views. +.. note:: + Thread safety is ensured only for OpenMP threads, and the |zfp| + views must be compiled by an OpenMP compliant compiler. As the + |zfp| compressed-array class implementation is defined in headers, + the application code using |zfp| must also be compiled with OpenMP + enabled if multithreaded access to |zfp| arrays is desired. + .. note:: Private views **do not guarantee cache coherence**. If, for example, the array is modified, then already cached data in a private view is From a3afa49f696865bdaf9aadc21adea4c9cf4baace Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 29 Apr 2023 12:40:41 -0700 Subject: [PATCH 168/277] Clarify intent of zfp_stream_maximum_size() [skip ci] --- docs/source/high-level-api.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index f47732869..7819bb410 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -555,6 +555,8 @@ Compressed Stream parameters stored in *stream* and the array whose scalar type and dimensions are given by *field*. This function may be used to determine how large a memory buffer to allocate to safely hold the entire compressed array. + The buffer may then be resized (using :code:`realloc()`) after the actual + number of bytes is known, as returned by :c:func:`zfp_compress`. .. _hl-func-stream: From 300e77d12f25d3eaa4ba9461d937fb17a71d45f6 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 29 Apr 2023 12:41:14 -0700 Subject: [PATCH 169/277] Clean up minor issues in documentation [skip ci] --- docs/source/arrays.rst | 2 +- docs/source/versions.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/arrays.rst b/docs/source/arrays.rst index 74f7dec07..31962e822 100644 --- a/docs/source/arrays.rst +++ b/docs/source/arrays.rst @@ -508,7 +508,7 @@ with only a few differences: - Whereas the constructors for fixed-rate arrays accept a *rate* parameter, the read-only arrays allow specifying any compression mode and - corresponding parameters (if any) via a :c:struct:`zfp_config` object. + corresponding parameters (if any) via a :c:type:`zfp_config` object. - Additional methods are available for setting and querying compression mode and parameters after construction. diff --git a/docs/source/versions.rst b/docs/source/versions.rst index d246adbbf..d42818654 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -7,7 +7,7 @@ Release Notes ------------------ This release is not ABI compatible with prior releases due to numerous changes -to function signatures and data structures like zfp_fieldu. However, few of +to function signatures and data structures like ``zfp_field``. However, few of the API changes, other than to the |cfp| C API for compressed arrays, should impact existing code. Note that numerous header files have been renamed or moved relative to prior versions. From c1845815ef1068b578b32823fb22310bd0b62b56 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 6 Aug 2023 19:26:28 -0700 Subject: [PATCH 170/277] Add void to functions taking no parameters (resolves #208) --- examples/diffusionC.c | 2 +- examples/iteratorC.c | 2 +- include/zfp.h | 6 +++--- include/zfp/bitstream.h | 2 +- include/zfp/bitstream.inl | 2 +- include/zfp/internal/cfp/array1d.h | 2 +- include/zfp/internal/cfp/array1f.h | 2 +- include/zfp/internal/cfp/array2d.h | 2 +- include/zfp/internal/cfp/array2f.h | 2 +- include/zfp/internal/cfp/array3d.h | 2 +- include/zfp/internal/cfp/array3f.h | 2 +- include/zfp/internal/cfp/array4d.h | 2 +- include/zfp/internal/cfp/array4f.h | 2 +- src/zfp.c | 6 +++--- utils/zfp.c | 2 +- 15 files changed, 19 insertions(+), 19 deletions(-) diff --git a/examples/diffusionC.c b/examples/diffusionC.c index 5e8404564..3a2ac6ab4 100644 --- a/examples/diffusionC.c +++ b/examples/diffusionC.c @@ -216,7 +216,7 @@ error(const double* u, const constants* c, double t) } static int -usage() +usage(void) { fprintf(stderr, "Usage: diffusionC [options]\n"); fprintf(stderr, "Options:\n"); diff --git a/examples/iteratorC.c b/examples/iteratorC.c index 8106c493a..93ef47259 100644 --- a/examples/iteratorC.c +++ b/examples/iteratorC.c @@ -30,7 +30,7 @@ void print3(cfp_iter1d begin, cfp_iter1d end) printf("%g\n", _.reference.get(_.iterator.ref(p))); } -int main() +int main(void) { const cfp_array1d_api _1d = cfp.array1d; const cfp_array2d_api _2d = cfp.array2d; diff --git a/include/zfp.h b/include/zfp.h index 9db4a3873..2d096cceb 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -335,7 +335,7 @@ zfp_stream_set_omp_chunk_size( /* unspecified configuration */ zfp_config /* compression mode and parameter settings */ -zfp_config_none(); +zfp_config_none(void); /* fixed-rate configuration */ zfp_config /* compression mode and parameter settings */ @@ -358,7 +358,7 @@ zfp_config_accuracy( /* reversible (lossless) configuration */ zfp_config /* compression mode and parameter settings */ -zfp_config_reversible(); +zfp_config_reversible(void); /* expert configuration */ zfp_config /* compression mode and parameter settings */ @@ -373,7 +373,7 @@ zfp_config_expert( /* allocate field struct */ zfp_field* /* pointer to default initialized field */ -zfp_field_alloc(); +zfp_field_alloc(void); /* allocate metadata for 1D field f[nx] */ zfp_field* /* allocated field metadata */ diff --git a/include/zfp/bitstream.h b/include/zfp/bitstream.h index 85922aadb..44598227e 100644 --- a/include/zfp/bitstream.h +++ b/include/zfp/bitstream.h @@ -34,7 +34,7 @@ void stream_close(bitstream* stream); bitstream* stream_clone(const bitstream* stream); /* word size in bits (equal to stream_word_bits) */ -bitstream_count stream_alignment(); +bitstream_count stream_alignment(void); /* pointer to beginning of stream */ void* stream_data(const bitstream* stream); diff --git a/include/zfp/bitstream.inl b/include/zfp/bitstream.inl index 987f5a42a..80294ee9e 100644 --- a/include/zfp/bitstream.inl +++ b/include/zfp/bitstream.inl @@ -171,7 +171,7 @@ stream_write_word(bitstream* s, bitstream_word value) /* word size in bits (equals bitstream_word_bits) */ inline_ bitstream_count -stream_alignment() +stream_alignment(void) { return wsize; } diff --git a/include/zfp/internal/cfp/array1d.h b/include/zfp/internal/cfp/array1d.h index 65bddff3b..fb20d3a77 100644 --- a/include/zfp/internal/cfp/array1d.h +++ b/include/zfp/internal/cfp/array1d.h @@ -96,7 +96,7 @@ typedef struct { } cfp_header1d_api; typedef struct { - cfp_array1d (*ctor_default)(); + cfp_array1d (*ctor_default)(void); cfp_array1d (*ctor)(size_t n, double rate, const double* p, size_t cache_size); cfp_array1d (*ctor_copy)(const cfp_array1d src); cfp_array1d (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array1f.h b/include/zfp/internal/cfp/array1f.h index 3f4d21e4f..6ca593d09 100644 --- a/include/zfp/internal/cfp/array1f.h +++ b/include/zfp/internal/cfp/array1f.h @@ -96,7 +96,7 @@ typedef struct { } cfp_header1f_api; typedef struct { - cfp_array1f (*ctor_default)(); + cfp_array1f (*ctor_default)(void); cfp_array1f (*ctor)(size_t n, double rate, const float* p, size_t cache_size); cfp_array1f (*ctor_copy)(const cfp_array1f src); cfp_array1f (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array2d.h b/include/zfp/internal/cfp/array2d.h index 9b070987d..b0e078af3 100644 --- a/include/zfp/internal/cfp/array2d.h +++ b/include/zfp/internal/cfp/array2d.h @@ -97,7 +97,7 @@ typedef struct { } cfp_header2d_api; typedef struct { - cfp_array2d (*ctor_default)(); + cfp_array2d (*ctor_default)(void); cfp_array2d (*ctor)(size_t nx, size_t ny, double rate, const double* p, size_t cache_size); cfp_array2d (*ctor_copy)(const cfp_array2d src); cfp_array2d (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array2f.h b/include/zfp/internal/cfp/array2f.h index 85bf584e7..0137b6094 100644 --- a/include/zfp/internal/cfp/array2f.h +++ b/include/zfp/internal/cfp/array2f.h @@ -97,7 +97,7 @@ typedef struct { } cfp_header2f_api; typedef struct { - cfp_array2f (*ctor_default)(); + cfp_array2f (*ctor_default)(void); cfp_array2f (*ctor)(size_t nx, size_t ny, double rate, const float* p, size_t cache_size); cfp_array2f (*ctor_copy)(const cfp_array2f src); cfp_array2f (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array3d.h b/include/zfp/internal/cfp/array3d.h index c3c337f6b..9c4a654a1 100644 --- a/include/zfp/internal/cfp/array3d.h +++ b/include/zfp/internal/cfp/array3d.h @@ -98,7 +98,7 @@ typedef struct { } cfp_header3d_api; typedef struct { - cfp_array3d (*ctor_default)(); + cfp_array3d (*ctor_default)(void); cfp_array3d (*ctor)(size_t nx, size_t ny, size_t nz, double rate, const double* p, size_t cache_size); cfp_array3d (*ctor_copy)(const cfp_array3d src); cfp_array3d (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array3f.h b/include/zfp/internal/cfp/array3f.h index 43fbe7229..e0f3aba5a 100644 --- a/include/zfp/internal/cfp/array3f.h +++ b/include/zfp/internal/cfp/array3f.h @@ -98,7 +98,7 @@ typedef struct { } cfp_header3f_api; typedef struct { - cfp_array3f (*ctor_default)(); + cfp_array3f (*ctor_default)(void); cfp_array3f (*ctor)(size_t nx, size_t ny, size_t nz, double rate, const float* p, size_t cache_size); cfp_array3f (*ctor_copy)(const cfp_array3f src); cfp_array3f (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array4d.h b/include/zfp/internal/cfp/array4d.h index 22e6a88fd..44d1ecf08 100644 --- a/include/zfp/internal/cfp/array4d.h +++ b/include/zfp/internal/cfp/array4d.h @@ -99,7 +99,7 @@ typedef struct { } cfp_header4d_api; typedef struct { - cfp_array4d (*ctor_default)(); + cfp_array4d (*ctor_default)(void); cfp_array4d (*ctor)(size_t nx, size_t ny, size_t nz, size_t nw, double rate, const double* p, size_t cache_size); cfp_array4d (*ctor_copy)(const cfp_array4d src); cfp_array4d (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/include/zfp/internal/cfp/array4f.h b/include/zfp/internal/cfp/array4f.h index b5e076748..b336dffe1 100644 --- a/include/zfp/internal/cfp/array4f.h +++ b/include/zfp/internal/cfp/array4f.h @@ -99,7 +99,7 @@ typedef struct { } cfp_header4f_api; typedef struct { - cfp_array4f (*ctor_default)(); + cfp_array4f (*ctor_default)(void); cfp_array4f (*ctor)(size_t nx, size_t ny, size_t nz, size_t nw, double rate, const float* p, size_t cache_size); cfp_array4f (*ctor_copy)(const cfp_array4f src); cfp_array4f (*ctor_header)(const cfp_header h, const void* buffer, size_t buffer_size_bytes); diff --git a/src/zfp.c b/src/zfp.c index a498a985f..6192bd614 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -105,7 +105,7 @@ zfp_type_size(zfp_type type) /* public functions: fields ------------------------------------------------ */ zfp_field* -zfp_field_alloc() +zfp_field_alloc(void) { zfp_field* field = (zfp_field*)malloc(sizeof(zfp_field)); if (field) { @@ -465,7 +465,7 @@ zfp_field_set_metadata(zfp_field* field, uint64 meta) /* public functions: compression mode and parameter settings --------------- */ zfp_config -zfp_config_none() +zfp_config_none(void) { zfp_config config; config.mode = zfp_mode_null; @@ -507,7 +507,7 @@ zfp_config_accuracy( } zfp_config -zfp_config_reversible() +zfp_config_reversible(void) { zfp_config config; config.mode = zfp_mode_reversible; diff --git a/utils/zfp.c b/utils/zfp.c index 3520f7437..1984532f5 100644 --- a/utils/zfp.c +++ b/utils/zfp.c @@ -80,7 +80,7 @@ print_error(const void* fin, const void* fout, zfp_type type, size_t n) } static void -usage() +usage(void) { fprintf(stderr, "%s\n", zfp_version_string); fprintf(stderr, "Usage: zfp \n"); From 2b523d92aec2d8c4d01b020045427c2e236b3910 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Wed, 23 Aug 2023 17:17:54 -0700 Subject: [PATCH 171/277] Removed unused functions from python test utils --- tests/python/test_utils.pyx | 23 ----------------------- 1 file changed, 23 deletions(-) diff --git a/tests/python/test_utils.pyx b/tests/python/test_utils.pyx index 6b486151b..e792e61e2 100644 --- a/tests/python/test_utils.pyx +++ b/tests/python/test_utils.pyx @@ -111,29 +111,6 @@ cdef extern from "zfpChecksums.h": zfp_type type, uint64_t key1, uint64_t key2) - uint64_t getChecksumOriginalDataBlock(int dims, - zfpy.zfp_type type) - uint64_t getChecksumEncodedBlock(int dims, - zfpy.zfp_type type) - uint64_t getChecksumEncodedPartialBlock(int dims, - zfpy.zfp_type type) - uint64_t getChecksumDecodedBlock(int dims, - zfpy.zfp_type type) - uint64_t getChecksumDecodedPartialBlock(int dims, - zfpy.zfp_type type) - uint64_t getChecksumOriginalDataArray(int ndims, - size_t[4] dims, - zfpy.zfp_type type) - uint64_t getChecksumCompressedBitstream(int ndims, - size_t[4] dims, - zfpy.zfp_type type, - zfpy.zfp_mode mode, - int compressParamNum) - uint64_t getChecksumDecompressedArray(int ndims, - size_t[4] dims, - zfpy.zfp_type ztype, - zfpy.zfp_mode mode, - int compressParamNum) cdef extern from "zfpHash.h": uint64_t hashBitstream(uint64_t* ptrStart, From d2a52e633a4b153b3db6a802b89c276d89f82410 Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 24 Aug 2023 12:06:26 -0700 Subject: [PATCH 172/277] update gitlab build options --- tests/gitlab/pascal-jobs.yml | 4 ++-- tests/gitlab/quartz-jobs.yml | 24 ++++++++++++------------ 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/tests/gitlab/pascal-jobs.yml b/tests/gitlab/pascal-jobs.yml index 1e5e5bead..151a186aa 100644 --- a/tests/gitlab/pascal-jobs.yml +++ b/tests/gitlab/pascal-jobs.yml @@ -4,9 +4,9 @@ cuda-10.1.168_build: variables: - ci_cmake: "cmake/3.9.2" + ci_cmake: "cmake/3.14.5" ci_cmp_mod: "cuda/10.1.168" - ci_gcc_mod: "gcc/8.3.1" + ci_gcc_mod: "gcc/10.3.1" extends: [.cuda, .pascal_build_gpu] needs: [] diff --git a/tests/gitlab/quartz-jobs.yml b/tests/gitlab/quartz-jobs.yml index 80591ba66..ca49faaa2 100644 --- a/tests/gitlab/quartz-jobs.yml +++ b/tests/gitlab/quartz-jobs.yml @@ -2,23 +2,23 @@ # CXX CPU # ########### -cpp_gnu-7.3.0_build: +cpp_gnu-10.3.1_build: variables: - ci_cmake: "cmake/3.9.2" + ci_cmake: "cmake/3.14.5" ci_cxx_cmp: "g++" ci_c_cmp: "gcc" - ci_cmp_mod: "gcc/7.3.0" + ci_cmp_mod: "gcc/10.3.1" extends: [.cpp, .quartz_build_cpu] needs: [] -cpp_gnu-7.3.0_test: +cpp_gnu-10.3.1_test: extends: [.quartz_test_cpu] - needs: [cpp_gnu-7.3.0_build] + needs: [cpp_gnu-10.3.1_build] cpp_clang-10.0.0_build: variables: - ci_cmake: "cmake/3.9.2" + ci_cmake: "cmake/3.14.5" ci_cxx_cmp: "clang++" ci_c_cmp: "clang" ci_cmp_mod: "clang/10.0.0" @@ -32,7 +32,7 @@ cpp_clang-10.0.0_test: cpp_intel-2022.3_build: variables: - ci_cmake: "cmake/3.9.2" + ci_cmake: "cmake/3.14.5" ci_cxx_cmp: "icpc" ci_c_cmp: "icc" ci_cmp_mod: "intel/2022.3" @@ -49,16 +49,16 @@ cpp_intel-2022.3_test: # C CPU # ######### -c_gnu-7.3.0_build: +c_gnu-10.3.1_build: variables: - ci_cmake: "cmake/3.9.2" + ci_cmake: "cmake/3.14.5" ci_c_cmp: "gcc" - ci_cmp_mod: "gcc/7.3.0" + ci_cmp_mod: "gcc/10.3.1" extends: [.c, .quartz_build_cpu] needs: [] -c_gnu-7.3.0_test: +c_gnu-10.3.1_test: variables: ci_test_regex: "Cfp" extends: [.quartz_test_cpu] - needs: [c_gnu-7.3.0_build] + needs: [c_gnu-10.3.1_build] From bc5e24145ea6f168cf5dd672e0f11544013004da Mon Sep 17 00:00:00 2001 From: gmorriso Date: Thu, 24 Aug 2023 13:04:02 -0700 Subject: [PATCH 173/277] update gitlab compiler versions --- tests/gitlab/gitlab-ci.yml | 1 - tests/gitlab/pascal-jobs.yml | 8 ++++---- tests/gitlab/quartz-jobs.yml | 16 ++++++++-------- 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 86fbf0e5f..bfd2345ea 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -73,7 +73,6 @@ stages: .build_gpu: before_script: - module reset - - module load opt - module load $ci_cmake - module load $ci_cmp_mod - module load $ci_gcc_mod diff --git a/tests/gitlab/pascal-jobs.yml b/tests/gitlab/pascal-jobs.yml index 151a186aa..7d73f17a7 100644 --- a/tests/gitlab/pascal-jobs.yml +++ b/tests/gitlab/pascal-jobs.yml @@ -2,16 +2,16 @@ # CUDA GPU # ############ -cuda-10.1.168_build: +cuda-11.8.0_build: variables: ci_cmake: "cmake/3.14.5" - ci_cmp_mod: "cuda/10.1.168" + ci_cmp_mod: "cuda/11.8.0" ci_gcc_mod: "gcc/10.3.1" extends: [.cuda, .pascal_build_gpu] needs: [] -cuda-10.1.168_test: +cuda-11.8.0_test: variables: ci_test_regex: "Cuda" extends: [.pascal_test_gpu] - needs: [cuda-10.1.168_build] + needs: [cuda-11.8.0_build] diff --git a/tests/gitlab/quartz-jobs.yml b/tests/gitlab/quartz-jobs.yml index ca49faaa2..672c68a59 100644 --- a/tests/gitlab/quartz-jobs.yml +++ b/tests/gitlab/quartz-jobs.yml @@ -16,32 +16,32 @@ cpp_gnu-10.3.1_test: needs: [cpp_gnu-10.3.1_build] -cpp_clang-10.0.0_build: +cpp_clang-14.0.6_build: variables: ci_cmake: "cmake/3.14.5" ci_cxx_cmp: "clang++" ci_c_cmp: "clang" - ci_cmp_mod: "clang/10.0.0" + ci_cmp_mod: "clang/14.0.6" extends: [.cpp, .quartz_build_cpu] needs: [] -cpp_clang-10.0.0_test: +cpp_clang-14.0.6_test: extends: [.quartz_test_cpu] - needs: [cpp_clang-10.0.0_build] + needs: [cpp_clang-14.0.6_build] -cpp_intel-2022.3_build: +cpp_intel-2022.1.0_build: variables: ci_cmake: "cmake/3.14.5" ci_cxx_cmp: "icpc" ci_c_cmp: "icc" - ci_cmp_mod: "intel/2022.3" + ci_cmp_mod: "intel/2022.1.0" extends: [.cpp, .quartz_build_cpu] needs: [] -cpp_intel-2022.3_test: +cpp_intel-2022.1.0_test: extends: [.quartz_test_cpu] - needs: [cpp_intel-2022.3_build] + needs: [cpp_intel-2022.1.0_build] From 6cdff306011d03ea76bd5e626a2f2d0ffb49b65b Mon Sep 17 00:00:00 2001 From: "Mark (he/his) C. Miller" Date: Sun, 27 Aug 2023 22:01:10 -0700 Subject: [PATCH 174/277] Add R&D100 badge I took a shot using shields.io ;) --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 1e25b75a4..ec5ada556 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@ ZFP [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) [![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.0)](https://zfp.readthedocs.io/en/release1.0.0/) [![codecov](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg?token=jqvMVvgRQ9)](https://codecov.io/gh/LLNL/zfp) +[![R&D100 - Winner](https://img.shields.io/badge/R%26D100-Winner-gold)](https://www.rdworldonline.com/rd-100-winners-for-2023-are-announced-2/) zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high From 920df4d9682f98615a6971a60988351662ac22ee Mon Sep 17 00:00:00 2001 From: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com> Date: Mon, 10 Oct 2022 14:34:45 +0200 Subject: [PATCH 175/277] Automatically update GitHub Actions in the future --- .github/dependabot.yml | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 .github/dependabot.yml diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 000000000..7bb4cf765 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,7 @@ +# https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file +version: 2 +updates: + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" From ad2f02b3e0dc3cfd18a30163d7503dc2944cf7ac Mon Sep 17 00:00:00 2001 From: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com> Date: Sat, 23 Sep 2023 19:28:41 +0200 Subject: [PATCH 176/277] Testing _WIN32 is enough, no need to test _WIN64 _WIN32 is defined on all Windows compilation targets, either 32-bit or 64-bit. It was initially supposed to distinguish between 16-bit and 32-bit targets, and is now defined by all compilers on both 32-bit and 64-bit Winodws compilation targets. https://learn.microsoft.com/en-us/cpp/preprocessor/predefined-macros?view=msvc-140 https://learn.microsoft.com/en-us/windows/win32/winprog64/the-tools Besides, this is consistent with the rest of the source fiels which only test _WIN32 to detect Windows compilation targets. --- include/zfp/internal/zfp/types.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/zfp/internal/zfp/types.h b/include/zfp/internal/zfp/types.h index b209f378e..5c8823673 100644 --- a/include/zfp/internal/zfp/types.h +++ b/include/zfp/internal/zfp/types.h @@ -66,7 +66,7 @@ typedef unsigned long ulong; typedef unsigned int uint32; /* determine 64-bit data model */ - #if defined(_WIN32) || defined(_WIN64) + #if defined(_WIN32) /* assume ILP32 or LLP64 (MSVC, MinGW) */ #define ZFP_LLP64 1 #else From 4c5d622ed4af9c91c4d1b3c81106af29c9bf30d7 Mon Sep 17 00:00:00 2001 From: Dimitri Papadopoulos <3234522+DimitriPapadopoulos@users.noreply.github.com> Date: Sat, 23 Sep 2023 19:25:10 +0200 Subject: [PATCH 177/277] Fix typo found by codespell --- docs/source/faq.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 77e20af2e..b03629931 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -968,7 +968,7 @@ and *q* = 55 + 4 |times| *d* |leq| 64 bits of precision for double-precision data. Of course, the constraint imposed by the available integer precision *n* implies that lossless compression of such data is possible only in 1D for single-precision data and only in 1D and 2D for double-precision data. -Finally, to preserve special values such as negative zero, plus and minues +Finally, to preserve special values such as negative zero, plus and minus infinity, and NaNs, reversible mode is needed. ------------------------------------------------------------------------------- From c61eca71d3ba93d8577974268b0122d473693177 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 29 Sep 2023 21:24:01 +0000 Subject: [PATCH 178/277] Bump actions/checkout from 3 to 4 Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 6f040669e..33c5b512f 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -11,7 +11,7 @@ jobs: if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: - name: Checkout - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Setup Python uses: actions/setup-python@v4 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 1c6a36b85..28dc2899d 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -7,7 +7,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout Zfp - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Setup Python uses: actions/setup-python@v4 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index 9170c521e..1ac394cfe 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -7,7 +7,7 @@ jobs: runs-on: macos-latest steps: - name: Checkout Zfp - uses: actions/checkout@v3 + uses: actions/checkout@v4 - name: Setup Python uses: actions/setup-python@v4 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 07c06974c..3d9dda098 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -37,7 +37,7 @@ jobs: target: all steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - name: Setup Python uses: actions/setup-python@v4 From 83c6e47ecadb95e8ed1c7e92a90129f82017f83d Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Sun, 1 Oct 2023 11:34:43 -0400 Subject: [PATCH 179/277] Update to support Python 3.12 --- python/scikit-build-cmake/FindNumPy.cmake | 33 ------------------- .../FindPythonExtensions.cmake | 10 ++---- 2 files changed, 3 insertions(+), 40 deletions(-) diff --git a/python/scikit-build-cmake/FindNumPy.cmake b/python/scikit-build-cmake/FindNumPy.cmake index cd78112b5..64066ada1 100644 --- a/python/scikit-build-cmake/FindNumPy.cmake +++ b/python/scikit-build-cmake/FindNumPy.cmake @@ -20,17 +20,6 @@ # # ``NumPy_INCLUDE_DIR`` # -# .. note:: -# -# To support NumPy < v0.15.0 where ``from-template`` and ``conv-template`` are not declared as entry points, -# the module emulates the behavior of standalone executables by setting the corresponding variables with the -# path the the python interpreter and the path to the associated script. For example: -# :: -# -# set(NumPy_CONV_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/conv_template.py CACHE STRING "Command executing conv-template program" FORCE) -# -# set(NumPy_FROM_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/from_template.py CACHE STRING "Command executing from-template program" FORCE) -# if(NOT NumPy_FOUND) set(_find_extra_args) @@ -59,28 +48,6 @@ if(NOT NumPy_FOUND) OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET ) - - # XXX This is required to support NumPy < v0.15.0. See note in module documentation above. - if(NOT NumPy_CONV_TEMPLATE_EXECUTABLE) - execute_process(COMMAND "${PYTHON_EXECUTABLE}" - -c "from numpy.distutils import conv_template; print(conv_template.__file__)" - OUTPUT_VARIABLE _numpy_conv_template_file - OUTPUT_STRIP_TRAILING_WHITESPACE - ERROR_QUIET - ) - set(NumPy_CONV_TEMPLATE_EXECUTABLE "${PYTHON_EXECUTABLE}" "${_numpy_conv_template_file}" CACHE STRING "Command executing conv-template program" FORCE) - endif() - - # XXX This is required to support NumPy < v0.15.0. See note in module documentation above. - if(NOT NumPy_FROM_TEMPLATE_EXECUTABLE) - execute_process(COMMAND "${PYTHON_EXECUTABLE}" - -c "from numpy.distutils import from_template; print(from_template.__file__)" - OUTPUT_VARIABLE _numpy_from_template_file - OUTPUT_STRIP_TRAILING_WHITESPACE - ERROR_QUIET - ) - set(NumPy_FROM_TEMPLATE_EXECUTABLE "${PYTHON_EXECUTABLE}" "${_numpy_from_template_file}" CACHE STRING "Command executing from-template program" FORCE) - endif() endif() endif() diff --git a/python/scikit-build-cmake/FindPythonExtensions.cmake b/python/scikit-build-cmake/FindPythonExtensions.cmake index 33e034d09..5a7a5ae9f 100644 --- a/python/scikit-build-cmake/FindPythonExtensions.cmake +++ b/python/scikit-build-cmake/FindPythonExtensions.cmake @@ -249,7 +249,7 @@ find_package(PythonLibs) include(targetLinkLibrariesWithDynamicLookup) set(_command " -import distutils.sysconfig +import sysconfig import itertools import os import os.path @@ -261,7 +261,7 @@ rel_result = None candidate_lists = [] try: - candidate_lists.append((distutils.sysconfig.get_python_lib(),)) + candidate_lists.append((sysconfig.get_paths()['purelib'],)) except AttributeError: pass try: @@ -282,17 +282,13 @@ for candidate in candidates: rel_result = rel_candidate break -ext_suffix_var = 'SO' -if sys.version_info[:2] >= (3, 5): - ext_suffix_var = 'EXT_SUFFIX' - sys.stdout.write(\";\".join(( os.sep, os.pathsep, sys.prefix, result, rel_result, - distutils.sysconfig.get_config_var(ext_suffix_var) + sysconfig.get_config_var('EXT_SUFFIX') ))) ") From bcc5a254823224c5010b51dc28d5c6c47b20ef39 Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Tue, 17 Oct 2023 11:25:41 -0700 Subject: [PATCH 180/277] Update gitlabCI runner config [skip ci] --- tests/gitlab/gitlab-ci.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index bfd2345ea..a544a6bcb 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -4,7 +4,8 @@ variables: GIT_SUBMODULE_STRATEGY: recursive - LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc -t 00:15:00" + LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc -t 00:20:00" + LLNL_SERVICE_USER: zfp stages: - build From d6bccbd7147d86288a593a6535b1fd94bf2349ed Mon Sep 17 00:00:00 2001 From: Garrett Morrison Date: Wed, 6 Dec 2023 12:36:27 -0800 Subject: [PATCH 181/277] Explicitly use old CMake FindPython policy --- python/CMakeLists.txt | 2 ++ python/scikit-build-cmake/FindCython.cmake | 2 ++ python/scikit-build-cmake/FindNumPy.cmake | 2 ++ python/scikit-build-cmake/FindPythonExtensions.cmake | 2 ++ python/scikit-build-cmake/UseCython.cmake | 2 ++ tests/python/CMakeLists.txt | 1 + 6 files changed, 11 insertions(+) diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index f2c1041b9..95a5c64e1 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -1,3 +1,5 @@ +cmake_policy(SET CMP0148 OLD) + set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_LIST_DIR}/scikit-build-cmake) include(UseCython) include(FindPythonExtensions) diff --git a/python/scikit-build-cmake/FindCython.cmake b/python/scikit-build-cmake/FindCython.cmake index 73534e9c9..8ab019db7 100644 --- a/python/scikit-build-cmake/FindCython.cmake +++ b/python/scikit-build-cmake/FindCython.cmake @@ -37,6 +37,8 @@ # Use the Cython executable that lives next to the Python executable # if it is a local installation. +cmake_policy(SET CMP0148 OLD) + find_package(PythonInterp) if(PYTHONINTERP_FOUND) execute_process(COMMAND ${PYTHON_EXECUTABLE} -c "from os import path; import cython; print(path.dirname(cython.__file__))" diff --git a/python/scikit-build-cmake/FindNumPy.cmake b/python/scikit-build-cmake/FindNumPy.cmake index cd78112b5..b1b770378 100644 --- a/python/scikit-build-cmake/FindNumPy.cmake +++ b/python/scikit-build-cmake/FindNumPy.cmake @@ -32,6 +32,8 @@ # set(NumPy_FROM_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/from_template.py CACHE STRING "Command executing from-template program" FORCE) # +cmake_policy(SET CMP0148 OLD) + if(NOT NumPy_FOUND) set(_find_extra_args) if(NumPy_FIND_REQUIRED) diff --git a/python/scikit-build-cmake/FindPythonExtensions.cmake b/python/scikit-build-cmake/FindPythonExtensions.cmake index 33e034d09..ffa205e76 100644 --- a/python/scikit-build-cmake/FindPythonExtensions.cmake +++ b/python/scikit-build-cmake/FindPythonExtensions.cmake @@ -244,6 +244,8 @@ # limitations under the License. #============================================================================= +cmake_policy(SET CMP0148 OLD) + find_package(PythonInterp REQUIRED) find_package(PythonLibs) include(targetLinkLibrariesWithDynamicLookup) diff --git a/python/scikit-build-cmake/UseCython.cmake b/python/scikit-build-cmake/UseCython.cmake index 2c40bd7b5..48a288f61 100644 --- a/python/scikit-build-cmake/UseCython.cmake +++ b/python/scikit-build-cmake/UseCython.cmake @@ -94,6 +94,8 @@ # limitations under the License. #============================================================================= +cmake_policy(SET CMP0148 OLD) + # Configuration options. set(CYTHON_ANNOTATE OFF CACHE BOOL "Create an annotated .html file when compiling *.pyx.") diff --git a/tests/python/CMakeLists.txt b/tests/python/CMakeLists.txt index a6b51eca0..815d6a987 100644 --- a/tests/python/CMakeLists.txt +++ b/tests/python/CMakeLists.txt @@ -1,3 +1,4 @@ +cmake_policy(SET CMP0148 OLD) set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_SOURCE_DIR}/python/scikit-build-cmake) find_package(PythonInterp REQUIRED) From e1441d8457bcdfe581f6948c2f647e2a506bcdc4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 11 Dec 2023 13:12:17 +0000 Subject: [PATCH 182/277] Bump actions/setup-python from 4 to 5 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 33c5b512f..f21bd891a 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -14,7 +14,7 @@ jobs: uses: actions/checkout@v4 - name: Setup Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 28dc2899d..776d6dadf 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -10,7 +10,7 @@ jobs: uses: actions/checkout@v4 - name: Setup Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index 1ac394cfe..cdc4cf40a 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -10,7 +10,7 @@ jobs: uses: actions/checkout@v4 - name: Setup Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 3d9dda098..a9f876ca5 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -40,7 +40,7 @@ jobs: - uses: actions/checkout@v4 - name: Setup Python - uses: actions/setup-python@v4 + uses: actions/setup-python@v5 with: python-version: '3.x' architecture: x64 From 63eeefa32adffe71217f3e48ae5011f8b5f75de1 Mon Sep 17 00:00:00 2001 From: Garrett Morrison <55602159+GarrettDMorrison@users.noreply.github.com> Date: Thu, 14 Dec 2023 12:43:19 -0800 Subject: [PATCH 183/277] Fix for issue #220 --- .github/workflows/debug-linux.yml | 1 + .github/workflows/debug-macos.yml | 1 + .github/workflows/tests.yml | 1 + CHANGELOG.md | 3 +- appveyor.yml | 11 +- python/CMakeLists.txt | 6 +- python/requirements.txt | 1 - python/scikit-build-cmake/FindCython.cmake | 21 +- python/scikit-build-cmake/FindNumPy.cmake | 37 +- .../FindPythonExtensions.cmake | 51 ++- python/scikit-build-cmake/UseCython.cmake | 20 +- .../UsePythonExtensions.cmake | 320 ++++++++++++++++++ ...targetLinkLibrariesWithDynamicLookup.cmake | 50 ++- setup.py | 2 +- tests/python/CMakeLists.txt | 7 +- 15 files changed, 461 insertions(+), 71 deletions(-) create mode 100644 python/scikit-build-cmake/UsePythonExtensions.cmake diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 28dc2899d..62268a50c 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -19,6 +19,7 @@ jobs: run: | python -m pip install cython python -m pip install oldest-supported-numpy + python -m pip install setuptools - name: Install OpenMP run: | diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index 1ac394cfe..36fbb976f 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -19,6 +19,7 @@ jobs: run: | python -m pip install cython python -m pip install oldest-supported-numpy + python -m pip install setuptools - name: Install OpenMP run: | diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 3d9dda098..51c56e9a7 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -49,6 +49,7 @@ jobs: run: | python -m pip install cython python -m pip install oldest-supported-numpy + python -m pip install setuptools - name: Setup OpenMP (Linux) if: ${{matrix.os == 'ubuntu-latest' && matrix.cxx_compiler == 'clang++'}} diff --git a/CHANGELOG.md b/CHANGELOG.md index 7387be1a9..c155595cc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,8 @@ Change Log ### Fixed -- # 177: Full test suite not in release +- #220: Errors reported with scikit-build when running `CMake` with `BUILD_ZFPY` enabled +- #177: Full test suite not in release - #176: `CFP` API is not exposed via CMake configuration file - #169: `libm` dependency is not always correctly detected. diff --git a/appveyor.yml b/appveyor.yml index abd9385af..18d3cb35b 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -8,26 +8,26 @@ environment: APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017 PLATFORM: x64 BUILD_TYPE: Release - PYTHON_VERSION: 35 + PYTHON_VERSION: 38 - COMPILER: msvc GENERATOR: Visual Studio 15 2017 APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017 PLATFORM: Win32 BUILD_TYPE: Release - PYTHON_VERSION: 35 + PYTHON_VERSION: 38 - COMPILER: msvc GENERATOR: Visual Studio 14 2015 Win64 PLATFORM: x64 BUILD_TYPE: Release - PYTHON_VERSION: 35 + PYTHON_VERSION: 38 - COMPILER: msvc GENERATOR: Visual Studio 14 2015 PLATFORM: Win32 BUILD_TYPE: Release - PYTHON_VERSION: 27 + PYTHON_VERSION: 38 - COMPILER: mingw GENERATOR: MinGW Makefiles @@ -63,5 +63,8 @@ install: - if "%COMPILER%"=="msvc" if "%BUILD_TYPE%"=="Release" pip install -r python\requirements.txt - if "%COMPILER%"=="msvc" if "%BUILD_TYPE%"=="Release" python --version + build_script: - sh appveyor.sh + # uncomment to enable interactive remote desktop mode + #- ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1')) diff --git a/python/CMakeLists.txt b/python/CMakeLists.txt index 95a5c64e1..9410ddd7f 100644 --- a/python/CMakeLists.txt +++ b/python/CMakeLists.txt @@ -1,4 +1,6 @@ -cmake_policy(SET CMP0148 OLD) +if (CMAKE_VERSION VERSION_GREATER_EQUAL 3.27.0) + cmake_policy(SET CMP0148 OLD) +endif () set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_LIST_DIR}/scikit-build-cmake) include(UseCython) @@ -14,7 +16,7 @@ find_package(NumPy REQUIRED) include_directories(${ZFP_SOURCE_DIR}/include) include_directories(${NumPy_INCLUDE_DIR}) -add_cython_target(zfpy zfpy.pyx C) +add_cython_target(zfpy zfpy.pyx C PY3) add_library(zfpy MODULE ${zfpy}) target_link_libraries(zfpy zfp) python_extension_module(zfpy) diff --git a/python/requirements.txt b/python/requirements.txt index 7f3612988..849962b23 100644 --- a/python/requirements.txt +++ b/python/requirements.txt @@ -1,3 +1,2 @@ cython>=0.22 numpy>=1.8.0 - diff --git a/python/scikit-build-cmake/FindCython.cmake b/python/scikit-build-cmake/FindCython.cmake index 8ab019db7..c8de13112 100644 --- a/python/scikit-build-cmake/FindCython.cmake +++ b/python/scikit-build-cmake/FindCython.cmake @@ -36,25 +36,22 @@ # Use the Cython executable that lives next to the Python executable # if it is a local installation. +if(Python_EXECUTABLE) + get_filename_component(_python_path ${Python_EXECUTABLE} PATH) +elseif(Python3_EXECUTABLE) + get_filename_component(_python_path ${Python3_EXECUTABLE} PATH) +elseif(DEFINED PYTHON_EXECUTABLE) + get_filename_component(_python_path ${PYTHON_EXECUTABLE} PATH) +endif() -cmake_policy(SET CMP0148 OLD) - -find_package(PythonInterp) -if(PYTHONINTERP_FOUND) - execute_process(COMMAND ${PYTHON_EXECUTABLE} -c "from os import path; import cython; print(path.dirname(cython.__file__))" - OUTPUT_VARIABLE CYTHON_PATH - ERROR_VARIABLE CYTHON_PATH - OUTPUT_STRIP_TRAILING_WHITESPACE - ERROR_STRIP_TRAILING_WHITESPACE) - +if(DEFINED _python_path) find_program(CYTHON_EXECUTABLE NAMES cython cython.bat cython3 - HINTS ${CYTHON_PATH} + HINTS ${_python_path} DOC "path to the cython executable") else() find_program(CYTHON_EXECUTABLE NAMES cython cython.bat cython3 - HINTS ${CYTHON_PATH} DOC "path to the cython executable") endif() diff --git a/python/scikit-build-cmake/FindNumPy.cmake b/python/scikit-build-cmake/FindNumPy.cmake index 69e948dc3..275ae1bee 100644 --- a/python/scikit-build-cmake/FindNumPy.cmake +++ b/python/scikit-build-cmake/FindNumPy.cmake @@ -20,8 +20,17 @@ # # ``NumPy_INCLUDE_DIR`` # - -cmake_policy(SET CMP0148 OLD) +# .. note:: +# +# To support NumPy < v0.15.0 where ``from-template`` and ``conv-template`` are not declared as entry points, +# the module emulates the behavior of standalone executables by setting the corresponding variables with the +# path the the python interpreter and the path to the associated script. For example: +# :: +# +# set(NumPy_CONV_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/conv_template.py CACHE STRING "Command executing conv-template program" FORCE) +# +# set(NumPy_FROM_TEMPLATE_EXECUTABLE /path/to/python /path/to/site-packages/numpy/distutils/from_template.py CACHE STRING "Command executing from-template program" FORCE) +# if(NOT NumPy_FOUND) set(_find_extra_args) @@ -31,8 +40,6 @@ if(NOT NumPy_FOUND) if(NumPy_FIND_QUIET) list(APPEND _find_extra_args QUIET) endif() - find_package(PythonInterp ${_find_extra_args}) - find_package(PythonLibs ${_find_extra_args}) find_program(NumPy_CONV_TEMPLATE_EXECUTABLE NAMES conv-template) find_program(NumPy_FROM_TEMPLATE_EXECUTABLE NAMES from-template) @@ -50,6 +57,28 @@ if(NOT NumPy_FOUND) OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET ) + + # XXX This is required to support NumPy < v0.15.0. See note in module documentation above. + if(NOT NumPy_CONV_TEMPLATE_EXECUTABLE) + execute_process(COMMAND "${PYTHON_EXECUTABLE}" + -c "from numpy.distutils import conv_template; print(conv_template.__file__)" + OUTPUT_VARIABLE _numpy_conv_template_file + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) + set(NumPy_CONV_TEMPLATE_EXECUTABLE "${PYTHON_EXECUTABLE}" "${_numpy_conv_template_file}" CACHE STRING "Command executing conv-template program" FORCE) + endif() + + # XXX This is required to support NumPy < v0.15.0. See note in module documentation above. + if(NOT NumPy_FROM_TEMPLATE_EXECUTABLE) + execute_process(COMMAND "${PYTHON_EXECUTABLE}" + -c "from numpy.distutils import from_template; print(from_template.__file__)" + OUTPUT_VARIABLE _numpy_from_template_file + OUTPUT_STRIP_TRAILING_WHITESPACE + ERROR_QUIET + ) + set(NumPy_FROM_TEMPLATE_EXECUTABLE "${PYTHON_EXECUTABLE}" "${_numpy_from_template_file}" CACHE STRING "Command executing from-template program" FORCE) + endif() endif() endif() diff --git a/python/scikit-build-cmake/FindPythonExtensions.cmake b/python/scikit-build-cmake/FindPythonExtensions.cmake index fbc3d04e1..59b30c2a2 100644 --- a/python/scikit-build-cmake/FindPythonExtensions.cmake +++ b/python/scikit-build-cmake/FindPythonExtensions.cmake @@ -168,8 +168,6 @@ # # .. code-block:: cmake # -# find_package(PythonInterp) -# find_package(PythonLibs) # find_package(PythonExtensions) # find_package(Cython) # find_package(Boost COMPONENTS python) @@ -244,14 +242,19 @@ # limitations under the License. #============================================================================= -cmake_policy(SET CMP0148 OLD) - find_package(PythonInterp REQUIRED) -find_package(PythonLibs) +if(SKBUILD AND NOT PYTHON_LIBRARY) + set(PYTHON_LIBRARY "no-library-required") + find_package(PythonLibs) + unset(PYTHON_LIBRARY) + unset(PYTHON_LIBRARIES) +else() + find_package(PythonLibs) +endif() include(targetLinkLibrariesWithDynamicLookup) set(_command " -import sysconfig +import distutils.sysconfig import itertools import os import os.path @@ -263,7 +266,7 @@ rel_result = None candidate_lists = [] try: - candidate_lists.append((sysconfig.get_paths()['purelib'],)) + candidate_lists.append((distutils.sysconfig.get_python_lib(),)) except AttributeError: pass try: @@ -290,7 +293,7 @@ sys.stdout.write(\";\".join(( sys.prefix, result, rel_result, - sysconfig.get_config_var('EXT_SUFFIX') + distutils.sysconfig.get_config_var('EXT_SUFFIX') ))) ") @@ -329,20 +332,38 @@ function(_set_python_extension_symbol_visibility _target) else() set(_modinit_prefix "init") endif() + message("_modinit_prefix:${_modinit_prefix}") if("${CMAKE_C_COMPILER_ID}" STREQUAL "MSVC") set_target_properties(${_target} PROPERTIES LINK_FLAGS "/EXPORT:${_modinit_prefix}${_target}" ) elseif("${CMAKE_C_COMPILER_ID}" STREQUAL "GNU" AND NOT ${CMAKE_SYSTEM_NAME} MATCHES "Darwin") + # Option to not run version script. See https://github.com/scikit-build/scikit-build/issues/668 + if(NOT DEFINED SKBUILD_GNU_SKIP_LOCAL_SYMBOL_EXPORT_OVERRIDE) + set(SKBUILD_GNU_SKIP_LOCAL_SYMBOL_EXPORT_OVERRIDE FALSE) + endif() set(_script_path ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/${_target}-version-script.map ) - file(WRITE ${_script_path} - "{global: ${_modinit_prefix}${_target}; local: *; };" - ) - set_property(TARGET ${_target} APPEND_STRING PROPERTY LINK_FLAGS + # Export all symbols. See https://github.com/scikit-build/scikit-build/issues/668 + if(SKBUILD_GNU_SKIP_LOCAL_SYMBOL_EXPORT_OVERRIDE) + file(WRITE ${_script_path} + "{global: ${_modinit_prefix}${_target};};" + ) + else() + file(WRITE ${_script_path} + "{global: ${_modinit_prefix}${_target}; local: *;};" + ) + endif() + if(NOT ${CMAKE_SYSTEM_NAME} MATCHES "SunOS") + set_property(TARGET ${_target} APPEND_STRING PROPERTY LINK_FLAGS " -Wl,--version-script=\"${_script_path}\"" - ) + ) + else() + set_property(TARGET ${_target} APPEND_STRING PROPERTY LINK_FLAGS + " -Wl,-M \"${_script_path}\"" + ) + endif() endif() endfunction() @@ -431,7 +452,7 @@ endfunction() function(python_standalone_executable _target) include_directories(${PYTHON_INCLUDE_DIRS}) - target_link_libraries(${_target} ${PYTHON_LIBRARIES}) + target_link_libraries(${_target} ${SKBUILD_LINK_LIBRARIES_KEYWORD} ${PYTHON_LIBRARIES}) endfunction() function(python_modules_header _name) @@ -572,3 +593,5 @@ function(python_modules_header _name) endif() set(${_include_dirs_var} ${CMAKE_CURRENT_BINARY_DIR} PARENT_SCOPE) endfunction() + +include(UsePythonExtensions) diff --git a/python/scikit-build-cmake/UseCython.cmake b/python/scikit-build-cmake/UseCython.cmake index 48a288f61..4e0fa7907 100644 --- a/python/scikit-build-cmake/UseCython.cmake +++ b/python/scikit-build-cmake/UseCython.cmake @@ -94,8 +94,6 @@ # limitations under the License. #============================================================================= -cmake_policy(SET CMP0148 OLD) - # Configuration options. set(CYTHON_ANNOTATE OFF CACHE BOOL "Create an annotated .html file when compiling *.pyx.") @@ -104,8 +102,6 @@ set(CYTHON_FLAGS "" CACHE STRING "Extra flags to the cython compiler.") mark_as_advanced(CYTHON_ANNOTATE CYTHON_FLAGS) -find_package(PythonLibs REQUIRED) - set(CYTHON_CXX_EXTENSION "cxx") set(CYTHON_C_EXTENSION "c") @@ -147,12 +143,6 @@ function(add_cython_target _name) message(FATAL_ERROR "Either C or CXX must be enabled to use Cython") endif() - if("${PYTHONLIBS_VERSION_STRING}" MATCHES "^2.") - set(_input_syntax "PY2") - else() - set(_input_syntax "PY3") - endif() - if(_args_EMBED_MAIN) set(_embed_main TRUE) endif() @@ -165,6 +155,10 @@ function(add_cython_target _name) set(_output_syntax "CXX") endif() + # Doesn't select an input syntax - Cython + # defaults to 2 for Cython 2 and 3 for Cython 3 + set(_input_syntax "default") + if(_args_PY2) set(_input_syntax "PY2") endif() @@ -210,15 +204,15 @@ function(add_cython_target _name) set(c_header_dependencies "") # Get the include directories. - get_source_file_property(pyx_location ${_source_file} LOCATION) - get_filename_component(pyx_path ${pyx_location} PATH) get_directory_property(cmake_include_directories - DIRECTORY ${pyx_path} + DIRECTORY ${CMAKE_CURRENT_LIST_DIR} INCLUDE_DIRECTORIES) list(APPEND cython_include_directories ${cmake_include_directories}) # Determine dependencies. # Add the pxd file with the same basename as the given pyx file. + get_source_file_property(pyx_location ${_source_file} LOCATION) + get_filename_component(pyx_path ${pyx_location} PATH) get_filename_component(pyx_file_basename ${_source_file} NAME_WE) unset(corresponding_pxd_file CACHE) find_file(corresponding_pxd_file ${pyx_file_basename}.pxd diff --git a/python/scikit-build-cmake/UsePythonExtensions.cmake b/python/scikit-build-cmake/UsePythonExtensions.cmake new file mode 100644 index 000000000..c411e20c4 --- /dev/null +++ b/python/scikit-build-cmake/UsePythonExtensions.cmake @@ -0,0 +1,320 @@ +#.rst: +# +# The following functions are defined: +# +# .. cmake:command:: add_python_library +# +# Add a library that contains a mix of C, C++, Fortran, Cython, F2PY, Template, +# and Tempita sources. The required targets are automatically generated to +# "lower" source files from their high-level representation to a file that the +# compiler can accept. +# +# +# add_python_library( +# SOURCES [source1 [source2 ...]] +# [INCLUDE_DIRECTORIES [dir1 [dir2 ...]] +# [LINK_LIBRARIES [lib1 [lib2 ...]] +# [DEPENDS [source1 [source2 ...]]]) +# +# +# Example usage +# ^^^^^^^^^^^^^ +# +# .. code-block:: cmake +# +# find_package(PythonExtensions) +# +# file(GLOB arpack_sources ARPACK/SRC/*.f ARPACK/UTIL/*.f) +# +# add_python_library(arpack_scipy +# SOURCES ${arpack_sources} +# ${g77_wrapper_sources} +# INCLUDE_DIRECTORIES ARPACK/SRC +# ) +# +# .. cmake:command:: add_python_extension +# +# Add a extension that contains a mix of C, C++, Fortran, Cython, F2PY, Template, +# and Tempita sources. The required targets are automatically generated to +# "lower" source files from their high-level representation to a file that the +# compiler can accept. +# +# +# add_python_extension( +# SOURCES [source1 [source2 ...]] +# [INCLUDE_DIRECTORIES [dir1 [dir2 ...]] +# [LINK_LIBRARIES [lib1 [lib2 ...]] +# [DEPENDS [source1 [source2 ...]]]) +# +# +# Example usage +# ^^^^^^^^^^^^^ +# +# .. code-block:: cmake +# +# find_package(PythonExtensions) +# +# file(GLOB arpack_sources ARPACK/SRC/*.f ARPACK/UTIL/*.f) +# +# add_python_extension(arpack_scipy +# SOURCES ${arpack_sources} +# ${g77_wrapper_sources} +# INCLUDE_DIRECTORIES ARPACK/SRC +# ) +# +# +#============================================================================= +# Copyright 2011 Kitware, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +#============================================================================= + +macro(_remove_whitespace _output) + string(REGEX REPLACE "[ \r\n\t]+" " " ${_output} "${${_output}}") + string(STRIP "${${_output}}" ${_output}) +endmacro() + +function(add_python_library _name) + set(options STATIC SHARED MODULE) + set(multiValueArgs SOURCES INCLUDE_DIRECTORIES LINK_LIBRARIES COMPILE_DEFINITIONS DEPENDS) + cmake_parse_arguments(_args "${options}" "" "${multiValueArgs}" ${ARGN} ) + + # Validate arguments to allow simpler debugging + if(NOT _args_SOURCES) + message( + FATAL_ERROR + "You have called add_python_library for library ${_name} without " + "any source files. This typically indicates a problem with " + "your CMakeLists.txt file" + ) + endif() + + # Initialize the list of sources + set(_sources ${_args_SOURCES}) + + # Generate targets for all *.src files + set(_processed ) + foreach(_source IN LISTS _sources) + if(${_source} MATCHES ".pyf.src$" OR ${_source} MATCHES "\\.f\\.src$") + if(NOT NumPy_FOUND) + message( + FATAL_ERROR + "NumPy is required to process *.src Template files" + ) + endif() + string(REGEX REPLACE "\\.[^.]*$" "" _source_we ${_source}) + add_custom_command( + OUTPUT ${_source_we} + COMMAND ${NumPy_FROM_TEMPLATE_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/${_source} + ${CMAKE_CURRENT_BINARY_DIR}/${_source_we} + DEPENDS ${_source} ${_args_DEPENDS} + COMMENT "Generating ${_source_we} from template ${_source}" + ) + list(APPEND _processed ${_source_we}) + elseif(${_source} MATCHES "\\.c\\.src$") + if(NOT NumPy_FOUND) + message( + FATAL_ERROR + "NumPy is required to process *.src Template files" + ) + endif() + string(REGEX REPLACE "\\.[^.]*$" "" _source_we ${_source}) + add_custom_command( + OUTPUT ${_source_we} + COMMAND ${NumPy_CONV_TEMPLATE_EXECUTABLE} + ${CMAKE_CURRENT_SOURCE_DIR}/${_source} + ${CMAKE_CURRENT_BINARY_DIR}/${_source_we} + DEPENDS ${_source} ${_args_DEPENDS} + COMMENT "Generating ${_source_we} from template ${_source}" + ) + list(APPEND _processed ${_source_we}) + elseif(${_source} MATCHES "\\.pyx\\.in$") + if(NOT Cython_FOUND) + message( + FATAL_ERROR + "Cython is required to process *.in Tempita files" + ) + endif() + string(REGEX REPLACE "\\.[^.]*$" "" _source_we ${_source}) + configure_file( + ${CMAKE_CURRENT_SOURCE_DIR}/${_source} + ${CMAKE_CURRENT_BINARY_DIR}/${_source} + COPYONLY + ) + set(_tempita_command + " + import os; + import sys; + from Cython.Tempita import Template; + cwd = os.getcwd(); + open(os.path.join(cwd, '${_source_we}'), 'w+') + .write( + Template.from_filename(os.path.join(cwd, '${_source}'), + encoding=sys.getdefaultencoding()).substitute() + ) + " + ) + _remove_whitespace(_tempita_command) + add_custom_command( + OUTPUT ${_source_we} + COMMAND ${PYTHON_EXECUTABLE} -c "${_tempita_command}" + DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/${_source}" + ${_args_DEPENDS} + ) + list(APPEND _processed ${_source_we}) + else() + list(APPEND _processed ${_source}) + endif() + endforeach() + set(_sources ${_processed}) + + # If we're building a Python extension and we're given only Fortran sources, + # We can conclude that we need to generate a Fortran interface file + list(FILTER _processed EXCLUDE REGEX "(\\.f|\\.f90)$") + if(NOT _processed AND _args_MODULE) + if(NOT NumPy_FOUND) + message( + FATAL_ERROR + "NumPy is required to process *.pyf F2PY files" + ) + endif() + set(_sources_abs ) + foreach(_source IN LISTS _sources) + if(NOT IS_ABSOLUTE ${_source}) + set(_source ${CMAKE_CURRENT_SOURCE_DIR}/${_source}) + endif() + list(APPEND _sources_abs ${_source}) + endforeach() + add_custom_command( + OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/${_name}.pyf + WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} + COMMAND ${F2PY_EXECUTABLE} + ARGS -h ${_name}.pyf -m ${_name} --overwrite-signature + ${_sources_abs} + DEPENDS ${_sources} ${_args_DEPENDS} + COMMENT "Generating ${_name} Fortran interface file" + ) + list(APPEND _sources ${_name}.pyf) + endif() + + # Are there F2PY targets? + set(_has_f2py_targets OFF) + set(_has_cython_targets OFF) + + # Generate targets for all *.pyx and *.pyf files + set(_processed ) + foreach(_source IN LISTS _sources) + if(${_source} MATCHES \\.pyx$) + if(NOT Cython_FOUND) + message( + FATAL_ERROR + "Cython is required to process *.pyx Cython files" + ) + endif() + string(REGEX REPLACE "\\.[^.]*$" "" _pyx_target_name ${_source}) + set(_has_cython_targets ON) + add_cython_target(${_pyx_target_name} + ${_source} + OUTPUT_VAR _pyx_target_output + DEPENDS ${_args_DEPENDS} + ) + list(APPEND _processed ${_pyx_target_output}) + elseif(${_source} MATCHES \\.pyf$) + if(NOT NumPy_FOUND) + message( + FATAL_ERROR + "NumPy is required to process *.pyf F2PY files" + ) + endif() + string(REGEX REPLACE "\\.[^.]*$" "" _pyf_target_name ${_source}) + set(_has_f2py_targets ON) + add_f2py_target(${_pyf_target_name} + ${_source} + OUTPUT_VAR _pyf_target_output + DEPENDS ${_args_DEPENDS} + ) + list(APPEND _processed ${_pyf_target_output}) + else() + list(APPEND _processed ${_source}) + endif() + endforeach() + set(_sources ${_processed}) + + if(_args_SHARED) + add_library(${_name} SHARED ${_sources}) + elseif(_args_MODULE) + add_library(${_name} MODULE ${_sources}) + else() + # Assume static + add_library(${_name} STATIC ${_sources}) + endif() + + target_include_directories(${_name} PRIVATE ${_args_INCLUDE_DIRECTORIES}) + target_link_libraries(${_name} ${SKBUILD_LINK_LIBRARIES_KEYWORD} ${_args_LINK_LIBRARIES}) + + if(_has_f2py_targets) + target_include_directories(${_name} PRIVATE ${F2PY_INCLUDE_DIRS}) + target_link_libraries(${_name} ${SKBUILD_LINK_LIBRARIES_KEYWORD} ${F2PY_LIBRARIES}) + endif() + + if(_args_COMPILE_DEFINITIONS) + target_compile_definitions(${_name} PRIVATE ${_args_COMPILE_DEFINITIONS}) + endif() + + if(_args_DEPENDS) + add_custom_target( + "${_name}_depends" + DEPENDS ${_args_DEPENDS} + ) + add_dependencies(${_name} "${_name}_depends") + endif() +endfunction() + +function(add_python_extension _name) + # FIXME: make sure that extensions with the same name can happen + # in multiple directories + + set(multiValueArgs SOURCES INCLUDE_DIRECTORIES LINK_LIBRARIES COMPILE_DEFINITIONS DEPENDS) + cmake_parse_arguments(_args "" "" "${multiValueArgs}" ${ARGN} ) + + # Validate arguments to allow simpler debugging + if(NOT _args_SOURCES) + message( + FATAL_ERROR + "You have called add_python_extension for library ${_name} without " + "any source files. This typically indicates a problem with " + "your CMakeLists.txt file" + ) + endif() + + add_python_library(${_name} MODULE + SOURCES ${_args_SOURCES} + INCLUDE_DIRECTORIES ${_args_INCLUDE_DIRECTORIES} + LINK_LIBRARIES ${_args_LINK_LIBRARIES} + COMPILE_DEFINITIONS ${_args_COMPILE_DEFINITIONS} + DEPENDS ${_args_DEPENDS} + ) + python_extension_module(${_name}) + + file(RELATIVE_PATH _relative "${CMAKE_SOURCE_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}") + if(_relative STREQUAL "") + set(_relative ".") + endif() + + install( + TARGETS ${_name} + LIBRARY DESTINATION "${_relative}" + RUNTIME DESTINATION "${_relative}" + ) +endfunction() diff --git a/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake b/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake index 6199ed5ea..a583f42cd 100644 --- a/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake +++ b/python/scikit-build-cmake/targetLinkLibrariesWithDynamicLookup.cmake @@ -198,6 +198,28 @@ function(_test_weak_link_project set(osx_dynamic_lookup "-undefined dynamic_lookup") set(no_flag "") + if(CMAKE_CROSSCOMPILING) + set(link_flag_spec "no_flag") + set(link_flag "${${link_flag_spec}}") + set(test_skipping_reason "") + set(test_pass FALSE) + + if(APPLE AND NOT CMAKE_CROSSCOMPILING_EMULATOR) + set(link_flag_spec "osx_dynamic_lookup") + set(link_flag "${${link_flag_spec}}") + set(test_skipping_reason " (Cross compiling without emulator on macOS)") + set(test_pass TRUE) + endif() + + if(test_pass) + set(test_description "Weak Link ${target_type} -> ${lib_type} (${link_flag_spec})") + message(STATUS "Performing Test ${test_description} - Assuming Success${test_skipping_reason}") + set(${can_weak_link_var} ${test_pass} PARENT_SCOPE) + set(${project_name} ${link_flag} PARENT_SCOPE) + return() + endif() + endif() + foreach(link_flag_spec gnu_ld_ignore osx_dynamic_lookup no_flag) set(link_flag "${${link_flag_spec}}") @@ -248,7 +270,7 @@ function(_test_weak_link_project if(link_mod_lib) file(APPEND "${test_project_src_dir}/CMakeLists.txt" " - target_link_libraries(counter number) + target_link_libraries(counter ${SKBUILD_LINK_LIBRARIES_KEYWORD} number) ") elseif(NOT link_flag STREQUAL "") file(APPEND "${test_project_src_dir}/CMakeLists.txt" " @@ -262,21 +284,21 @@ function(_test_weak_link_project if(link_exe_lib) file(APPEND "${test_project_src_dir}/CMakeLists.txt" " - target_link_libraries(main number) + target_link_libraries(main ${SKBUILD_LINK_LIBRARIES_KEYWORD} number) ") elseif(NOT link_flag STREQUAL "") file(APPEND "${test_project_src_dir}/CMakeLists.txt" " - target_link_libraries(main \"${link_flag}\") + target_link_libraries(main ${SKBUILD_LINK_LIBRARIES_KEYWORD} \"${link_flag}\") ") endif() if(link_exe_mod) file(APPEND "${test_project_src_dir}/CMakeLists.txt" " - target_link_libraries(main counter) + target_link_libraries(main ${SKBUILD_LINK_LIBRARIES_KEYWORD} counter) ") else() file(APPEND "${test_project_src_dir}/CMakeLists.txt" " - target_link_libraries(main \"${CMAKE_DL_LIBS}\") + target_link_libraries(main ${SKBUILD_LINK_LIBRARIES_KEYWORD} \"${CMAKE_DL_LIBS}\") ") endif() @@ -492,21 +514,15 @@ function(_check_dynamic_lookup endif() if(NOT DEFINED ${cache_var}) - set(skip_test FALSE) - if(CMAKE_CROSSCOMPILING AND NOT CMAKE_CROSSCOMPILING_EMULATOR) + if(CMAKE_CROSSCOMPILING AND NOT CMAKE_CROSSCOMPILING_EMULATOR) set(skip_test TRUE) endif() - if(skip_test) - set(has_dynamic_lookup FALSE) - set(link_flags) - else() - _test_weak_link_project(${target_type} - ${lib_type} - has_dynamic_lookup - link_flags) - endif() + _test_weak_link_project(${target_type} + ${lib_type} + has_dynamic_lookup + link_flags) set(caveat " (when linking ${target_type} against ${lib_type})") @@ -576,6 +592,6 @@ function(target_link_libraries_with_dynamic_lookup target) set(links "${link_items}" "${link_libs}") if(links) - target_link_libraries(${target} "${links}") + target_link_libraries(${target} ${SKBUILD_LINK_LIBRARIES_KEYWORD} "${links}") endif() endfunction() diff --git a/setup.py b/setup.py index 9e4c39119..668e52392 100644 --- a/setup.py +++ b/setup.py @@ -11,5 +11,5 @@ long_description="zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high throughput read and write random access to individual array elements. zfp also supports serial and parallel compression of whole arrays using both lossless and lossy compression with error tolerances. zfp is primarily written in C and C++ but also includes Python and Fortran bindings.", ext_modules=[Extension("zfpy", ["build/python/zfpy.c"], include_dirs=["include", np.get_include()], - libraries=["zfp"], library_dirs=["build/lib64", "build/lib/Release"])] + libraries=["zfp"], library_dirs=["build/lib64", "build/lib/Release"]), language_level = "3"] ) diff --git a/tests/python/CMakeLists.txt b/tests/python/CMakeLists.txt index 815d6a987..589ac8afa 100644 --- a/tests/python/CMakeLists.txt +++ b/tests/python/CMakeLists.txt @@ -1,4 +1,7 @@ -cmake_policy(SET CMP0148 OLD) +if (CMAKE_VERSION VERSION_GREATER_EQUAL 3.27.0) + cmake_policy(SET CMP0148 OLD) +endif () + set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_SOURCE_DIR}/python/scikit-build-cmake) find_package(PythonInterp REQUIRED) @@ -14,7 +17,7 @@ include_directories(${NumPy_INCLUDE_DIR}) include_directories(${ZFP_SOURCE_DIR}/tests/python) include_directories(${ZFP_SOURCE_DIR}/tests/utils) include_directories(${ZFP_SOURCE_DIR}) -add_cython_target(test_utils test_utils.pyx C) +add_cython_target(test_utils test_utils.pyx C PY3) add_library(test_utils MODULE ${test_utils}) target_link_libraries(test_utils zfp genSmoothRandNumsLib stridedOperationsLib zfpCompressionParamsLib zfpChecksumsLib zfpHashLib) python_extension_module(test_utils) From c81062e2b2776e89bf820127e8a2df91cfc8cd1e Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 5 Dec 2023 16:48:10 -0800 Subject: [PATCH 184/277] Clarify that 64-bit OS is not required (resolves #194) --- docs/source/installation.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/installation.rst b/docs/source/installation.rst index fb11314b8..f4eceaa4f 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -20,7 +20,7 @@ core library can also be built using GNU make on Linux, macOS, and MinGW. C++11, and C++14. .. note:: - |zfp| requires 64-bit compiler and operating system support. + |zfp| requires compiler support for 64-bit integers. .. _cmake_builds: From e892d713bafd02603efe3c01a56de9ba7bbed5ef Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 6 Dec 2023 12:45:40 -0800 Subject: [PATCH 185/277] Document that OS=mac is needed when building dylibs with gmake (resolves #170) --- docs/source/installation.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/installation.rst b/docs/source/installation.rst index f4eceaa4f..23de28ae3 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -193,6 +193,8 @@ Regardless of the settings below, |libzfp| will always be built. CMake default: on. GNU make default: off. +.. note:: + On macOS, add :code:`OS=mac` when building shared libraries with GNU make. .. index:: single: Configuration From 5f3ade7e24b18755d304ec8d95632671653b7833 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 6 Dec 2023 15:09:17 -0800 Subject: [PATCH 186/277] Fix broken links --- docs/source/introduction.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index c2f180f3c..56c924640 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -57,11 +57,12 @@ bindings) to install. |zfp| is also available through several package managers, including Conda (both `C/C++ `__ and `Python `__ packages are available), -`PIP `__, and -`Spack `__. -`RPM packages `__ are available -for several Linux distributions and may be installed using :code:`apt` or -:code:`yum`. +`PIP `__, +`Spack `__, and +`MacPorts `__. +`Linux packages `__ are available +for several distributions and may be installed, for example, using :code:`apt` +and :code:`yum`. .. _app-support: @@ -83,7 +84,7 @@ plugins, and formats, such as select compressors shipped with `HDF5 binaries `__. -* `Compression functions `__ +* `Compression functions `__ for Intel\ |reg| `Integrated Performance Primitives `__. * `Compressed MPI messages `__ From 113767ee376a353cad7fdff2db7e7fcfb97dc8ac Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 14 Dec 2023 15:37:24 -0800 Subject: [PATCH 187/277] Add missing resolved issues to CHANGELOG --- CHANGELOG.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c155595cc..a958038ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,12 +5,21 @@ Change Log ## Unreleased +### Added + +- A new build macro, `BUILD_TESTING_FULL`, specifies that all unit tests be + built; `BUILD_TESTING` produces a smaller subset of tests. Full tests and + documentation are now included in releases. + ### Fixed -- #220: Errors reported with scikit-build when running `CMake` with `BUILD_ZFPY` enabled -- #177: Full test suite not in release -- #176: `CFP` API is not exposed via CMake configuration file - #169: `libm` dependency is not always correctly detected. +- #171: `ptrdiff_t` is not always imported in Cython. +- #176: `CFP` API is not exposed via CMake configuration file. +- #177: Full test suite is not included in release. +- #181: `rpath` is not set correctly in executables. +- #204: Array strides are not passed by value in `zFORp`. +- #220: Errors reported with scikit-build when running `CMake` with `BUILD_ZFPY` enabled. --- From c5879e22255f842811f38c7e51b5d34781bcec2b Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 14 Dec 2023 15:49:30 -0800 Subject: [PATCH 188/277] Mention R&D 100 award --- docs/source/introduction.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst index 56c924640..2757eab0f 100644 --- a/docs/source/introduction.rst +++ b/docs/source/introduction.rst @@ -41,6 +41,8 @@ exactly, is also supported. `Lawrence Livermore National Laboratory `__ and is supported by the U.S. Department of Energy's `Exascale Computing Project `__. +|zfp| is a +`2023 R&D 100 Award Winner `__. Availability From 325bba6ca057434b894a2b60b58181168258ad43 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 15 Dec 2023 14:24:52 -0800 Subject: [PATCH 189/277] Add RTD v2 yaml file --- .readthedocs.yaml | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 .readthedocs.yaml diff --git a/.readthedocs.yaml b/.readthedocs.yaml new file mode 100644 index 000000000..b5814d22f --- /dev/null +++ b/.readthedocs.yaml @@ -0,0 +1,35 @@ +# Read the Docs configuration file for Sphinx projects +# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details + +# Required +version: 2 + +# Set the OS, Python version and other tools you might need +build: + os: ubuntu-22.04 + tools: + python: "3.12" + # You can also specify other tool versions: + # nodejs: "20" + # rust: "1.70" + # golang: "1.20" + +# Build documentation in the "docs/" directory with Sphinx +sphinx: + configuration: docs/source/conf.py + # You can configure Sphinx to use a different builder, for instance use the dirhtml builder for simpler URLs + # builder: "dirhtml" + # Fail on all warnings to avoid broken references + # fail_on_warning: true + +# Optionally build your docs in additional formats such as PDF and ePub +formats: + - pdf +# - epub + +# Optional but recommended, declare the Python requirements required +# to build your documentation +# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html +# python: +# install: +# - requirements: docs/requirements.txt From a0bab67395090e1302ca2b698d3de7556a005966 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 15 Dec 2023 14:49:10 -0800 Subject: [PATCH 190/277] Add sphinx-fortran dependency --- .readthedocs.yaml | 6 +++--- docs/requirements.txt | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) create mode 100644 docs/requirements.txt diff --git a/.readthedocs.yaml b/.readthedocs.yaml index b5814d22f..b6421ddd6 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -30,6 +30,6 @@ formats: # Optional but recommended, declare the Python requirements required # to build your documentation # See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html -# python: -# install: -# - requirements: docs/requirements.txt +python: + install: + - requirements: docs/requirements.txt diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 000000000..682da2168 --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1 @@ +sphinx-fortran==1.1.1 From 913e67fa735890744263b339ae44b8d1bef15be6 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 15 Dec 2023 14:59:54 -0800 Subject: [PATCH 191/277] Add dependency on six --- docs/requirements.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/requirements.txt b/docs/requirements.txt index 682da2168..f0c7424af 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1 +1,4 @@ sphinx-fortran==1.1.1 + +# required by sphinx-fortran but not installed on RTD +six From 6bdada5f36bb5d774f909e63fb5e697198391fe6 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 15 Dec 2023 16:50:02 -0800 Subject: [PATCH 192/277] Set release version and date --- CHANGELOG.md | 11 +++++++---- LICENSE | 2 +- README.md | 6 +++--- appveyor.yml | 2 +- docs/source/conf.py | 13 ++++++++----- docs/source/contributors.rst | 2 +- docs/source/license.rst | 2 +- docs/source/versions.rst | 24 ++++++++++++++++++++++++ fortran/zfp.f90 | 6 +++--- include/zfp.h | 2 +- include/zfp.hpp | 2 +- include/zfp/version.h | 5 +---- setup.py | 4 ++-- src/zfp.c | 2 +- 14 files changed, 55 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a958038ac..96aa92169 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,10 @@ Change Log --- -## Unreleased +## 1.0.1 (2023-12-15) + +This patch release primarily addresses minor bug fixes and is needed to update +the zfpy Python wheels. ### Added @@ -15,11 +18,11 @@ Change Log - #169: `libm` dependency is not always correctly detected. - #171: `ptrdiff_t` is not always imported in Cython. -- #176: `CFP` API is not exposed via CMake configuration file. +- #176: cfp API is not exposed via CMake configuration file. - #177: Full test suite is not included in release. - #181: `rpath` is not set correctly in executables. -- #204: Array strides are not passed by value in `zFORp`. -- #220: Errors reported with scikit-build when running `CMake` with `BUILD_ZFPY` enabled. +- #204: Array strides are not passed by value in zFORp. +- #220: Errors reported with scikit-build when building zfpy. --- diff --git a/LICENSE b/LICENSE index 9bed13fb4..3c726d295 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ BSD 3-Clause License -Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC +Copyright (c) 2014-2023, Lawrence Livermore National Security, LLC All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/README.md b/README.md index ec5ada556..c053aba67 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ZFP === [![Github Actions Build Status](https://github.com/LLNL/zfp/workflows/Run%20Tests/badge.svg)](https://github.com/LLNL/zfp/actions/workflows/tests.yml) [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) -[![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.0)](https://zfp.readthedocs.io/en/release1.0.0/) +[![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.1)](https://zfp.readthedocs.io/en/release1.0.1/) [![codecov](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg?token=jqvMVvgRQ9)](https://codecov.io/gh/LLNL/zfp) [![R&D100 - Winner](https://img.shields.io/badge/R%26D100-Winner-gold)](https://www.rdworldonline.com/rd-100-winners-for-2023-are-announced-2/) @@ -67,9 +67,9 @@ section. Documentation ------------- -Full HTML [documentation](http://zfp.readthedocs.io/en/release1.0.0) is +Full HTML [documentation](http://zfp.readthedocs.io/en/release1.0.1) is available online. -A [PDF](http://readthedocs.org/projects/zfp/downloads/pdf/release1.0.0/) +A [PDF](http://readthedocs.org/projects/zfp/downloads/pdf/release1.0.1/) version is also available. Further information on the zfp software is included in these files: diff --git a/appveyor.yml b/appveyor.yml index 18d3cb35b..8a1e4688b 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -1,4 +1,4 @@ -version: 1.0.0-{build} +version: 1.0.1-{build} environment: # zfpy only build for Release builds (otherwise need debug python libs python27_d.lib) diff --git a/docs/source/conf.py b/docs/source/conf.py index b669ef5bf..0536c39e6 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -46,8 +46,8 @@ # General information about the project. project = u'zfp' -copyright = u'2014-2022, LLNL-CODE-663824' -author = u'Peter Lindstrom, Garrett Morrison' +copyright = u'2014-2023, LLNL-CODE-663824' +author = u'Peter Lindstrom, Danielle Asher' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -56,14 +56,17 @@ # The short X.Y version. version = u'1.0' # The full version, including alpha/beta/rc tags. -release = u'1.0.0' +release = u'1.0.1' + +# The release date (as the RTD server is in another time zone). +today = u'Dec 15, 2023' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # # This is also used if you do content translation via gettext catalogs. # Usually you set "language" from the command line for these cases. -language = None +language = 'en' # Enable automatic numbering of figures referenced by :numref:. numfig = True @@ -149,7 +152,7 @@ # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'zfp.tex', u'zfp Documentation', - u'\shortstack[l]{Peter Lindstrom\\\\Garrett Morrison}', 'manual'), + u'\shortstack[l]{Peter Lindstrom\\\\Danielle Asher}', 'manual'), ] diff --git a/docs/source/contributors.rst b/docs/source/contributors.rst index 53d33fbf9..45e945c24 100644 --- a/docs/source/contributors.rst +++ b/docs/source/contributors.rst @@ -7,7 +7,7 @@ Contributors * |zfp| development team - Peter Lindstrom - - Garrett Morrison + - Danielle Asher * Major contributors diff --git a/docs/source/license.rst b/docs/source/license.rst index 0c8d6e5ce..1129b833f 100644 --- a/docs/source/license.rst +++ b/docs/source/license.rst @@ -3,7 +3,7 @@ License ======= -| Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC +| Copyright (c) 2014-2023, Lawrence Livermore National Security, LLC | All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/docs/source/versions.rst b/docs/source/versions.rst index d42818654..0824e55cd 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -3,6 +3,30 @@ Release Notes ============= +1.0.1 (2023-12-15) +------------------ + +This patch release primarily addresses minor bug fixes and is needed to update +the zfpy Python wheels. + +**Added** + +- A new build macro, ``BUILD_TESTING_FULL``, specifies that all unit tests be + built; ``BUILD_TESTING`` produces a smaller subset of tests. Full tests and + documentation are now included in releases. + +**Fixed** + +- #169: `libm` dependency is not always correctly detected. +- #171: `ptrdiff_t` is not always imported in Cython. +- #176: cfp API is not exposed via CMake configuration file. +- #177: Full test suite is not included in release. +- #181: `rpath` is not set correctly in executables. +- #204: Array strides are not passed by value in zFORp. +- #220: Errors reported with scikit-build when building zfpy. + +---- + 1.0.0 (2022-08-01) ------------------ diff --git a/fortran/zfp.f90 b/fortran/zfp.f90 index 8418b0e51..ae5714685 100644 --- a/fortran/zfp.f90 +++ b/fortran/zfp.f90 @@ -53,7 +53,7 @@ module zfp integer, parameter :: const_zFORp_version_major = 1 integer, parameter :: const_zFORp_version_minor = 0 - integer, parameter :: const_zFORp_version_patch = 0 + integer, parameter :: const_zFORp_version_patch = 1 integer, parameter :: const_zFORp_version_tweak = 0 integer, protected, bind(c, name="zFORp_version_major") :: zFORp_version_major integer, protected, bind(c, name="zFORp_version_minor") :: zFORp_version_minor @@ -68,11 +68,11 @@ module zfp integer, protected, bind(c, name="zFORp_codec_version") :: zFORp_codec_version data zFORp_codec_version/const_zFORp_codec_version/ - integer, parameter :: const_zFORp_library_version = 4096 ! 0x1000 + integer, parameter :: const_zFORp_library_version = 4112 ! 0x1010 integer, protected, bind(c, name="zFORp_library_version") :: zFORp_library_version data zFORp_library_version/const_zFORp_library_version/ - character(len = 36), parameter :: zFORp_version_string = 'zfp version 1.0.0 (August 1, 2022)' + character(len = 36), parameter :: zFORp_version_string = 'zfp version 1.0.1 (December 15, 2023)' integer, parameter :: const_zFORp_min_bits = 1 integer, parameter :: const_zFORp_max_bits = 16658 diff --git a/include/zfp.h b/include/zfp.h index 2d096cceb..5e87a3244 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -1,5 +1,5 @@ /* -** Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC and +** Copyright (c) 2014-2023, Lawrence Livermore National Security, LLC and ** other zfp project contributors. See the top-level LICENSE file for details. ** SPDX-License-Identifier: BSD-3-Clause */ diff --git a/include/zfp.hpp b/include/zfp.hpp index 406fec9bb..5ec93fd48 100644 --- a/include/zfp.hpp +++ b/include/zfp.hpp @@ -1,7 +1,7 @@ #ifndef ZFP_HPP #define ZFP_HPP -// Copyright (c) 2014-2022, Lawrence Livermore National Security, LLC and +// Copyright (c) 2014-2023, Lawrence Livermore National Security, LLC and // other zfp project contributors. See the top-level LICENSE file for details. // SPDX-License-Identifier: BSD-3-Clause diff --git a/include/zfp/version.h b/include/zfp/version.h index 75872a282..1390bb36d 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -4,12 +4,9 @@ /* library version information */ #define ZFP_VERSION_MAJOR 1 /* library major version number */ #define ZFP_VERSION_MINOR 0 /* library minor version number */ -#define ZFP_VERSION_PATCH 0 /* library patch version number */ +#define ZFP_VERSION_PATCH 1 /* library patch version number */ #define ZFP_VERSION_TWEAK 0 /* library tweak version number */ -/* defined for work in progress (indicates unofficial release) */ -#define ZFP_VERSION_DEVELOP 1 - /* codec version number (see also zfp_codec_version) */ #define ZFP_CODEC 5 diff --git a/setup.py b/setup.py index 668e52392..fa2da6efc 100644 --- a/setup.py +++ b/setup.py @@ -3,8 +3,8 @@ setup( name="zfpy", - version="1.0.0", - author="Peter Lindstrom", + version="1.0.1", + author="Peter Lindstrom, Danielle Asher", author_email="zfp@llnl.gov", url="https://zfp.llnl.gov", description="zfp compression in Python", diff --git a/src/zfp.c b/src/zfp.c index 6192bd614..6aa8b8f67 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -11,7 +11,7 @@ const uint zfp_codec_version = ZFP_CODEC; const uint zfp_library_version = ZFP_VERSION; -const char* const zfp_version_string = "zfp version " ZFP_VERSION_STRING " (August 1, 2022)"; +const char* const zfp_version_string = "zfp version " ZFP_VERSION_STRING " (December 15, 2023)"; /* private functions ------------------------------------------------------- */ From cd174e0bfe36344f5ff5747bbfbf4be44380be61 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 15 Dec 2023 17:21:28 -0800 Subject: [PATCH 193/277] Bump version number --- README.md | 4 ++-- docs/source/installation.rst | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index c053aba67..5ee0ef636 100644 --- a/README.md +++ b/README.md @@ -58,9 +58,9 @@ Note: GNU builds are less flexible and do not support all available features, e.g., CUDA support. For further configuration and build instructions, please consult the -[documentation](https://zfp.readthedocs.io/en/release1.0.0/installation.html). +[documentation](https://zfp.readthedocs.io/en/release1.0.1/installation.html). For examples of how to call the C library and use the C++ array classes, -see the [examples](https://zfp.readthedocs.io/en/release1.0.0/examples.html) +see the [examples](https://zfp.readthedocs.io/en/release1.0.1/examples.html) section. diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 23de28ae3..9009fa891 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -30,7 +30,7 @@ CMake Builds To build |zfp| using `CMake `__ on Linux or macOS, start a Unix shell and type:: - cd zfp-1.0.0 + cd zfp-1.0.1 mkdir build cd build cmake .. @@ -48,7 +48,7 @@ By default, CMake builds will attempt to locate and use To build |zfp| using Visual Studio on Windows, start a DOS shell and type:: - cd zfp-1.0.0 + cd zfp-1.0.1 mkdir build cd build cmake .. @@ -67,7 +67,7 @@ GNU Builds To build |zfp| using `gcc `__ without `OpenMP `__, type:: - cd zfp-1.0.0 + cd zfp-1.0.1 gmake This builds |libzfp| as a static library as well as the |zfp| From 58abb28d842ada6fb446e857ae77a0a7987783cf Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 11 Jan 2024 10:05:26 -0800 Subject: [PATCH 194/277] Correct stride types in revcodec --- include/zfp/version.h | 3 +++ src/template/revdecode.c | 2 +- src/template/revencode.c | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/zfp/version.h b/include/zfp/version.h index 1390bb36d..272802d3f 100644 --- a/include/zfp/version.h +++ b/include/zfp/version.h @@ -7,6 +7,9 @@ #define ZFP_VERSION_PATCH 1 /* library patch version number */ #define ZFP_VERSION_TWEAK 0 /* library tweak version number */ +/* defined for work in progress (indicates unofficial release) */ +#define ZFP_VERSION_DEVELOP 1 + /* codec version number (see also zfp_codec_version) */ #define ZFP_CODEC 5 diff --git a/src/template/revdecode.c b/src/template/revdecode.c index 115b0a177..30ec1f76a 100644 --- a/src/template/revdecode.c +++ b/src/template/revdecode.c @@ -4,7 +4,7 @@ static void _t2(rev_inv_xform, Int, DIMS)(Int* p); /* reversible inverse lifting transform of 4-vector */ static void -_t1(rev_inv_lift, Int)(Int* p, uint s) +_t1(rev_inv_lift, Int)(Int* p, ptrdiff_t s) { Int x, y, z, w; x = *p; p += s; diff --git a/src/template/revencode.c b/src/template/revencode.c index fa1621406..f68df269a 100644 --- a/src/template/revencode.c +++ b/src/template/revencode.c @@ -4,7 +4,7 @@ static void _t2(rev_fwd_xform, Int, DIMS)(Int* p); /* reversible forward lifting transform of 4-vector */ static void -_t1(rev_fwd_lift, Int)(Int* p, uint s) +_t1(rev_fwd_lift, Int)(Int* p, ptrdiff_t s) { Int x, y, z, w; x = *p; p += s; From f61aac5ca3445ac48bab94d0e3da62bb77bdd989 Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 11 Jan 2024 10:10:56 -0800 Subject: [PATCH 195/277] Remove extraneous pointer casts --- src/share/parallel.c | 2 +- src/template/compress.c | 6 +++--- src/template/decompress.c | 6 +++--- src/template/ompcompress.c | 8 ++++---- src/zfp.c | 8 ++++---- utils/zfp.c | 16 ++++++++-------- 6 files changed, 23 insertions(+), 23 deletions(-) diff --git a/src/share/parallel.c b/src/share/parallel.c index 1ae365268..2f407be50 100644 --- a/src/share/parallel.c +++ b/src/share/parallel.c @@ -49,7 +49,7 @@ compress_init_par(zfp_stream* stream, const zfp_field* field, size_t chunks, siz (stream_wtell(stream->stream) % stream_word_bits != 0); /* set up buffer for each thread to compress to */ - bs = (bitstream**)malloc(chunks * sizeof(bitstream*)); + bs = malloc(chunks * sizeof(bitstream*)); if (!bs) return NULL; for (chunk = 0; chunk < chunks; chunk++) { diff --git a/src/template/compress.c b/src/template/compress.c index 74983c56b..dca6a34a1 100644 --- a/src/template/compress.c +++ b/src/template/compress.c @@ -2,7 +2,7 @@ static void _t2(compress, Scalar, 1)(zfp_stream* stream, const zfp_field* field) { - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; size_t mx = nx & ~3u; size_t x; @@ -37,7 +37,7 @@ _t2(compress_strided, Scalar, 1)(zfp_stream* stream, const zfp_field* field) static void _t2(compress_strided, Scalar, 2)(zfp_stream* stream, const zfp_field* field) { - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; size_t ny = field->ny; ptrdiff_t sx = field->sx ? field->sx : 1; @@ -59,7 +59,7 @@ _t2(compress_strided, Scalar, 2)(zfp_stream* stream, const zfp_field* field) static void _t2(compress_strided, Scalar, 3)(zfp_stream* stream, const zfp_field* field) { - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; size_t ny = field->ny; size_t nz = field->nz; diff --git a/src/template/decompress.c b/src/template/decompress.c index 726107737..19f02abdc 100644 --- a/src/template/decompress.c +++ b/src/template/decompress.c @@ -2,7 +2,7 @@ static void _t2(decompress, Scalar, 1)(zfp_stream* stream, zfp_field* field) { - Scalar* data = (Scalar*)field->data; + Scalar* data = field->data; size_t nx = field->nx; size_t mx = nx & ~3u; size_t x; @@ -37,7 +37,7 @@ _t2(decompress_strided, Scalar, 1)(zfp_stream* stream, zfp_field* field) static void _t2(decompress_strided, Scalar, 2)(zfp_stream* stream, zfp_field* field) { - Scalar* data = (Scalar*)field->data; + Scalar* data = field->data; size_t nx = field->nx; size_t ny = field->ny; ptrdiff_t sx = field->sx ? field->sx : 1; @@ -59,7 +59,7 @@ _t2(decompress_strided, Scalar, 2)(zfp_stream* stream, zfp_field* field) static void _t2(decompress_strided, Scalar, 3)(zfp_stream* stream, zfp_field* field) { - Scalar* data = (Scalar*)field->data; + Scalar* data = field->data; size_t nx = field->nx; size_t ny = field->ny; size_t nz = field->nz; diff --git a/src/template/ompcompress.c b/src/template/ompcompress.c index 4e4365c72..ca446dbda 100644 --- a/src/template/ompcompress.c +++ b/src/template/ompcompress.c @@ -5,7 +5,7 @@ static void _t2(compress_omp, Scalar, 1)(zfp_stream* stream, const zfp_field* field) { /* array metadata */ - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; /* number of omp threads, blocks, and chunks */ @@ -52,7 +52,7 @@ static void _t2(compress_strided_omp, Scalar, 1)(zfp_stream* stream, const zfp_field* field) { /* array metadata */ - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; ptrdiff_t sx = field->sx ? field->sx : 1; @@ -100,7 +100,7 @@ static void _t2(compress_strided_omp, Scalar, 2)(zfp_stream* stream, const zfp_field* field) { /* array metadata */ - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; size_t ny = field->ny; ptrdiff_t sx = field->sx ? field->sx : 1; @@ -155,7 +155,7 @@ static void _t2(compress_strided_omp, Scalar, 3)(zfp_stream* stream, const zfp_field* field) { /* array metadata */ - const Scalar* data = (const Scalar*)field->data; + const Scalar* data = field->data; size_t nx = field->nx; size_t ny = field->ny; size_t nz = field->nz; diff --git a/src/zfp.c b/src/zfp.c index 6aa8b8f67..5d763784f 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -107,7 +107,7 @@ zfp_type_size(zfp_type type) zfp_field* zfp_field_alloc(void) { - zfp_field* field = (zfp_field*)malloc(sizeof(zfp_field)); + zfp_field* field = malloc(sizeof(zfp_field)); if (field) { field->type = zfp_type_none; field->nx = field->ny = field->nz = field->nw = 0; @@ -189,7 +189,7 @@ zfp_field_begin(const zfp_field* field) if (field->data) { ptrdiff_t min; field_index_span(field, &min, NULL); - return (void*)((uchar*)field->data + min * (ptrdiff_t)zfp_type_size(field->type)); + return (uchar*)field->data + min * (ptrdiff_t)zfp_type_size(field->type); } else return NULL; @@ -536,7 +536,7 @@ zfp_config_expert( zfp_stream* zfp_stream_open(bitstream* stream) { - zfp_stream* zfp = (zfp_stream*)malloc(sizeof(zfp_stream)); + zfp_stream* zfp = malloc(sizeof(zfp_stream)); if (zfp) { zfp->stream = stream; zfp->minbits = ZFP_MIN_BITS; @@ -941,7 +941,7 @@ zfp_stream_set_execution(zfp_stream* zfp, zfp_exec_policy policy) zfp_exec_params_omp* params = malloc(sizeof(zfp_exec_params_omp)); params->threads = 0; params->chunk_size = 0; - zfp->exec.params = (void*)params; + zfp->exec.params = params; } break; #else diff --git a/utils/zfp.c b/utils/zfp.c index 1984532f5..9744f896a 100644 --- a/utils/zfp.c +++ b/utils/zfp.c @@ -30,14 +30,14 @@ The 7 major tasks to be accomplished are: static void print_error(const void* fin, const void* fout, zfp_type type, size_t n) { - const int32* i32i = (const int32*)fin; - const int64* i64i = (const int64*)fin; - const float* f32i = (const float*)fin; - const double* f64i = (const double*)fin; - const int32* i32o = (const int32*)fout; - const int64* i64o = (const int64*)fout; - const float* f32o = (const float*)fout; - const double* f64o = (const double*)fout; + const int32* i32i = fin; + const int64* i64i = fin; + const float* f32i = fin; + const double* f64i = fin; + const int32* i32o = fout; + const int64* i64o = fout; + const float* f32o = fout; + const double* f64o = fout; double fmin = +DBL_MAX; double fmax = -DBL_MAX; double erms = 0; From 260e1545e8d2648c383cc01d9e322191c88c3eb0 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 5 Feb 2024 13:48:20 +0000 Subject: [PATCH 196/277] Bump codecov/codecov-action from 3 to 4 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3 to 4. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/v3...v4) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 33c5b512f..54c4313cd 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -42,7 +42,7 @@ jobs: lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml - name: Upload Report to Codecov - uses: codecov/codecov-action@v3 + uses: codecov/codecov-action@v4 with: files: ${{github.workspace}}/build/coverage.xml env_vars: Actions From 6814a4ebbbee6002f1d7f39a0acc9e8429545bce Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 20 Feb 2024 16:03:27 -0800 Subject: [PATCH 197/277] Add fallthrough_ macro for switch statements --- examples/inplace.c | 2 +- examples/speed.c | 2 +- include/zfp/internal/zfp/system.h | 15 +++++++++++++++ src/cuda_zfp/encode.cuh | 8 ++++---- src/template/encode.c | 8 ++++---- src/zfp.c | 12 ++++++------ tests/array/utils/gtestBaseFixture.h | 2 ++ tests/src/endtoend/ompExecBase.c | 3 +++ tests/src/endtoend/serialExecBase.c | 4 ++++ tests/testviews.cpp | 4 ++-- tests/utils/genSmoothRandNums.c | 5 +++++ 11 files changed, 47 insertions(+), 18 deletions(-) diff --git a/examples/inplace.c b/examples/inplace.c index 9516240d0..67d1b3e4d 100644 --- a/examples/inplace.c +++ b/examples/inplace.c @@ -97,7 +97,7 @@ int main(int argc, char* argv[]) case 2: if (sscanf(argv[1], "%lf", &tolerance) != 1) goto usage; - /* FALLTHROUGH */ + fallthrough_ case 1: break; default: diff --git a/examples/speed.c b/examples/speed.c index e75f42854..ea4de02bf 100644 --- a/examples/speed.c +++ b/examples/speed.c @@ -91,7 +91,7 @@ int main(int argc, char* argv[]) switch (argc) { case 3: sscanf(argv[2], "%u", &blocks); - /* FALLTHROUGH */ + fallthrough_ case 2: sscanf(argv[1], "%lf", &rate); break; diff --git a/include/zfp/internal/zfp/system.h b/include/zfp/internal/zfp/system.h index 23c493600..9aa4d3c37 100644 --- a/include/zfp/internal/zfp/system.h +++ b/include/zfp/internal/zfp/system.h @@ -1,6 +1,7 @@ #ifndef ZFP_SYSTEM_H #define ZFP_SYSTEM_H +/* restrict keyword */ #if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99: use restrict */ #define restrict_ restrict @@ -9,6 +10,20 @@ #define restrict_ #endif +/* fallthrough in switch statements */ +#define fallthrough_ /* FALLTHROUGH */ +#if defined(__cplusplus) && __cplusplus >= 201703L + /* C++17: use [[fallthrough]] */ + #undef fallthrough_ + #define fallthrough_ [[fallthrough]]; +#elif defined(__has_attribute) + #if __has_attribute(fallthrough) + /* GNUC: use __attribute__((fallthrough)) */ + #undef fallthrough_ + #define fallthrough_ __attribute__((fallthrough)); + #endif +#endif + /* macros for exporting and importing symbols */ #if defined(_MSC_VER) && defined(ZFP_SHARED_LIBS) /* export (import) symbols when ZFP_SOURCE is (is not) defined */ diff --git a/src/cuda_zfp/encode.cuh b/src/cuda_zfp/encode.cuh index 995c9c321..cf8c8c9e5 100644 --- a/src/cuda_zfp/encode.cuh +++ b/src/cuda_zfp/encode.cuh @@ -22,16 +22,16 @@ void pad_block(Scalar *p, uint n, uint s) { case 0: p[0 * s] = 0; - /* FALLTHROUGH */ + fallthrough_ case 1: p[1 * s] = p[0 * s]; - /* FALLTHROUGH */ + fallthrough_ case 2: p[2 * s] = p[1 * s]; - /* FALLTHROUGH */ + fallthrough_ case 3: p[3 * s] = p[0 * s]; - /* FALLTHROUGH */ + fallthrough_ default: break; } diff --git a/src/template/encode.c b/src/template/encode.c index c085a4ab2..027bd44f4 100644 --- a/src/template/encode.c +++ b/src/template/encode.c @@ -11,16 +11,16 @@ _t1(pad_block, Scalar)(Scalar* p, size_t n, ptrdiff_t s) switch (n) { case 0: p[0 * s] = 0; - /* FALLTHROUGH */ + fallthrough_ case 1: p[1 * s] = p[0 * s]; - /* FALLTHROUGH */ + fallthrough_ case 2: p[2 * s] = p[1 * s]; - /* FALLTHROUGH */ + fallthrough_ case 3: p[3 * s] = p[0 * s]; - /* FALLTHROUGH */ + fallthrough_ default: break; } diff --git a/src/zfp.c b/src/zfp.c index 5d763784f..f13a4025a 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -220,13 +220,13 @@ zfp_field_size(const zfp_field* field, size_t* size) switch (zfp_field_dimensionality(field)) { case 4: size[3] = field->nw; - /* FALLTHROUGH */ + fallthrough_ case 3: size[2] = field->nz; - /* FALLTHROUGH */ + fallthrough_ case 2: size[1] = field->ny; - /* FALLTHROUGH */ + fallthrough_ case 1: size[0] = field->nx; break; @@ -263,13 +263,13 @@ zfp_field_stride(const zfp_field* field, ptrdiff_t* stride) switch (zfp_field_dimensionality(field)) { case 4: stride[3] = field->sw ? field->sw : (ptrdiff_t)(field->nx * field->ny * field->nz); - /* FALLTHROUGH */ + fallthrough_ case 3: stride[2] = field->sz ? field->sz : (ptrdiff_t)(field->nx * field->ny); - /* FALLTHROUGH */ + fallthrough_ case 2: stride[1] = field->sy ? field->sy : (ptrdiff_t)field->nx; - /* FALLTHROUGH */ + fallthrough_ case 1: stride[0] = field->sx ? field->sx : 1; break; diff --git a/tests/array/utils/gtestBaseFixture.h b/tests/array/utils/gtestBaseFixture.h index 82b6be9a3..d63e09de6 100644 --- a/tests/array/utils/gtestBaseFixture.h +++ b/tests/array/utils/gtestBaseFixture.h @@ -65,12 +65,14 @@ class CArrayNdTestFixture : public ::testing::TestWithParam { config = zfp_config_reversible(); break; } +#if 0 case zfp_mode_expert: { //TODO: do we need this one? //config = zfp_config_expert(uint minbits, uint maxbits, uint maxprec, int minexp); //break; } +#endif default: { config = zfp_config_none(); diff --git a/tests/src/endtoend/ompExecBase.c b/tests/src/endtoend/ompExecBase.c index 986286630..926f61a27 100644 --- a/tests/src/endtoend/ompExecBase.c +++ b/tests/src/endtoend/ompExecBase.c @@ -20,10 +20,13 @@ computeTotalBlocks(zfp_field* field) switch (zfp_field_dimensionality(field)) { case 4: bw = (field->nw + 3) / 4; + fallthrough_ case 3: bz = (field->nz + 3) / 4; + fallthrough_ case 2: by = (field->ny + 3) / 4; + fallthrough_ case 1: bx = (field->nx + 3) / 4; return bx * by * bz * bw; diff --git a/tests/src/endtoend/serialExecBase.c b/tests/src/endtoend/serialExecBase.c index 067957b11..28d7f861b 100644 --- a/tests/src/endtoend/serialExecBase.c +++ b/tests/src/endtoend/serialExecBase.c @@ -112,12 +112,16 @@ isCompressedBitrateComparableToChosenRate(struct setupVars* bundle) switch (DIMS) { case 4: paddedArrayLen *= paddedNw; + fallthrough_ case 3: paddedArrayLen *= paddedNz; + fallthrough_ case 2: paddedArrayLen *= paddedNy; + fallthrough_ case 1: paddedArrayLen *= paddedNx; + break; } // expect bitrate to scale wrt padded array length diff --git a/tests/testviews.cpp b/tests/testviews.cpp index fc2b17219..bc3d4b3b6 100644 --- a/tests/testviews.cpp +++ b/tests/testviews.cpp @@ -56,13 +56,13 @@ int main(int argc, char* argv[]) (std::istringstream(argv[8]) >> my).fail() || !my || (std::istringstream(argv[9]) >> mz).fail() || !mz) return usage(); - // FALLTHROUGH + fallthrough_ case 4: if ((std::istringstream(argv[1]) >> nx).fail() || !nx || (std::istringstream(argv[2]) >> ny).fail() || !ny || (std::istringstream(argv[3]) >> nz).fail() || !nz) return usage(); - // FALLTHROUGH + fallthrough_ case 1: break; default: diff --git a/tests/utils/genSmoothRandNums.c b/tests/utils/genSmoothRandNums.c index 20c2ba47a..1c12ca280 100644 --- a/tests/utils/genSmoothRandNums.c +++ b/tests/utils/genSmoothRandNums.c @@ -5,6 +5,7 @@ #include "genSmoothRandNums.h" #include "fixedpoint96.h" #include "rand64.h" +#include "zfp.h" #define FLOAT_MANTISSA_BITS 23 #define DOUBLE_MANTISSA_BITS 52 @@ -30,15 +31,19 @@ computeOffset(size_t l, size_t k, size_t j, size_t i, size_t sideLen, int numDim switch (numDims) { case 4: result += l * sideLen * sideLen * sideLen; + fallthrough_ case 3: result += k * sideLen * sideLen; + fallthrough_ case 2: result += j * sideLen; + fallthrough_ case 1: result += i; + break; } return result; From c936a20a85a70b15f574caf7fc79262e78b2140c Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 17 May 2024 18:50:08 -0700 Subject: [PATCH 198/277] Add GitLab directive required by LC --- tests/gitlab/gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index a544a6bcb..8a4c5a85b 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -135,6 +135,8 @@ stages: ############ include: + - project: 'lc-templates/id_tokens' + file: 'id_tokens.yml' - local: tests/gitlab/pascal-templates.yml - local: tests/gitlab/pascal-jobs.yml - local: tests/gitlab/quartz-templates.yml From 6aa2dae1c1bf700f062f386e81cc71796929c30e Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 17 May 2024 22:14:34 -0700 Subject: [PATCH 199/277] Fix Actions badge [skip ci] --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5ee0ef636..4688473cb 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ ZFP === -[![Github Actions Build Status](https://github.com/LLNL/zfp/workflows/Run%20Tests/badge.svg)](https://github.com/LLNL/zfp/actions/workflows/tests.yml) +[![GitHub Actions Test Status](https://github.com/LLNL/zfp/actions/workflows/tests.yml/badge.svg)](https://github.com/LLNL/zfp/actions/workflows/tests.yml) [![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/qb3ld7j11segy52k/branch/develop?svg=true)](https://ci.appveyor.com/project/lindstro/zfp) [![Documentation Status](https://readthedocs.org/projects/zfp/badge/?version=release1.0.1)](https://zfp.readthedocs.io/en/release1.0.1/) [![codecov](https://codecov.io/gh/LLNL/zfp/branch/develop/graph/badge.svg?token=jqvMVvgRQ9)](https://codecov.io/gh/LLNL/zfp) From 5c976d8da013988174f931845862b6f94119cade Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 29 Jun 2024 08:33:54 -0700 Subject: [PATCH 200/277] Run tests on pull request --- .github/workflows/tests.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 51c56e9a7..fce4abb37 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -1,6 +1,6 @@ name: Tests -on: push +on: [push, pull_request] env: BUILD_TYPE: Release From 37004a11e24a383d82b762dc44e542ddbd93ce6c Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Wed, 3 Jul 2024 15:26:38 -0400 Subject: [PATCH 201/277] fix: setup.py + set language level for cython --- python/zfpy.pxd | 2 ++ python/zfpy.pyx | 2 ++ setup.py | 24 ++++++++++++++++++++---- 3 files changed, 24 insertions(+), 4 deletions(-) diff --git a/python/zfpy.pxd b/python/zfpy.pxd index c92c29833..067bd90a4 100644 --- a/python/zfpy.pxd +++ b/python/zfpy.pxd @@ -1,3 +1,5 @@ +# cython: language_level=3 + import cython cimport libc.stdint as stdint from libc.stddef cimport ptrdiff_t diff --git a/python/zfpy.pyx b/python/zfpy.pyx index 0b7a19e1c..0d1eb5230 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -1,3 +1,5 @@ +# cython: language_level=3 + import sys import operator import functools diff --git a/setup.py b/setup.py index fa2da6efc..544ae2918 100644 --- a/setup.py +++ b/setup.py @@ -1,15 +1,31 @@ from setuptools import setup, Extension -import numpy as np + +class NumpyImport: + def __repr__(self): + import numpy as np + + return np.get_include() + + __fspath__ = __repr__ setup( name="zfpy", + setup_requires=["numpy", "cython"], version="1.0.1", author="Peter Lindstrom, Danielle Asher", author_email="zfp@llnl.gov", url="https://zfp.llnl.gov", description="zfp compression in Python", long_description="zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high throughput read and write random access to individual array elements. zfp also supports serial and parallel compression of whole arrays using both lossless and lossy compression with error tolerances. zfp is primarily written in C and C++ but also includes Python and Fortran bindings.", - ext_modules=[Extension("zfpy", ["build/python/zfpy.c"], - include_dirs=["include", np.get_include()], - libraries=["zfp"], library_dirs=["build/lib64", "build/lib/Release"]), language_level = "3"] + ext_modules=[ + Extension( + "zfpy", + sources=["python/zfpy.pyx"], + include_dirs=["include", str(NumpyImport())], + libraries=["zfp"], + library_dirs=["build/lib64", "build/lib/Release"], + language_level=3, + lanugage="c", + ), + ], ) From 923f7c6ccab15e3820d8893226e44b9711d59a19 Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Wed, 3 Jul 2024 15:30:14 -0400 Subject: [PATCH 202/277] chore: add trove classifiers --- setup.py | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/setup.py b/setup.py index 544ae2918..90b37865c 100644 --- a/setup.py +++ b/setup.py @@ -15,6 +15,7 @@ def __repr__(self): author="Peter Lindstrom, Danielle Asher", author_email="zfp@llnl.gov", url="https://zfp.llnl.gov", + license="License :: OSI Approved :: BSD License", description="zfp compression in Python", long_description="zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high throughput read and write random access to individual array elements. zfp also supports serial and parallel compression of whole arrays using both lossless and lossy compression with error tolerances. zfp is primarily written in C and C++ but also includes Python and Fortran bindings.", ext_modules=[ @@ -28,4 +29,20 @@ def __repr__(self): lanugage="c", ), ], + classifiers=[ + "Intended Audience :: Developers", + "Development Status :: 4 - Beta", + "License :: OSI Approved :: BSD License", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Topic :: Scientific/Engineering :: Image Processing", + "Topic :: System :: Archiving :: Compression", + "Operating System :: POSIX", + "Operating System :: MacOS", + "Operating System :: Microsoft :: Windows :: Windows 10", + ], ) From 3b11d3169170e7125414fee1e4bb3f8ad753648d Mon Sep 17 00:00:00 2001 From: William Silversmith Date: Wed, 3 Jul 2024 15:41:18 -0400 Subject: [PATCH 203/277] install: add MANIFEST.in and pyproject.toml --- MANIFEST.in | 6 ++++++ pyproject.toml | 8 ++++++++ 2 files changed, 14 insertions(+) create mode 100644 MANIFEST.in create mode 100644 pyproject.toml diff --git a/MANIFEST.in b/MANIFEST.in new file mode 100644 index 000000000..a34c60c68 --- /dev/null +++ b/MANIFEST.in @@ -0,0 +1,6 @@ +include python/zfpy.pxd +include python/zfpy.pyx +recursive-include include *.h +recursive-include src *.c *.h +include LICENSE +include pyproject.toml \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 000000000..f43c81a75 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,8 @@ +[build-system] +requires = [ + "setuptools", + "wheel", + "cython", + "oldest-supported-numpy; python_version<'3.9'", + 'numpy; python_version>="3.9"', +] \ No newline at end of file From 7ea1c3432c59e4c27443a341e5b99ea1ed69a7cc Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 22 Aug 2024 09:48:31 -0700 Subject: [PATCH 204/277] Document that compress/decompress must use same field dimensions --- docs/source/faq.rst | 94 +++++++++++++++++++++++++--------- docs/source/high-level-api.rst | 32 +++++++++--- 2 files changed, 94 insertions(+), 32 deletions(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index b03629931..35908b600 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -498,31 +498,48 @@ information independently. Q15: *Must I use the same parameters during compression and decompression?* -A: Not necessarily. When decompressing one block at a time, it is possible -to use more tightly constrained :c:type:`zfp_stream` parameters during -decompression than were used during compression. For instance, one may use a -smaller :c:member:`zfp_stream.maxbits`, -smaller :c:member:`zfp_stream.maxprec`, or larger :c:member:`zfp_stream.minexp` -during decompression to process fewer compressed bits than are stored, and to -decompress the array more quickly at a lower precision. This may be useful -in situations where the precision and accuracy requirements are not known a -priori, thus forcing conservative settings during compression, or when the -compressed stream is used for multiple purposes. For instance, visualization -usually has less stringent precision requirements than quantitative data -analysis. This feature of decompressing to a lower precision is particularly -useful when the stream is stored progressively (see :ref:`Q13 `). - -Note that one may not use less constrained parameters during decompression, -e.g., one cannot ask for more than :c:member:`zfp_stream.maxprec` bits of -precision when decompressing. Furthermore, the parameters must agree between -compression and decompression when calling the high-level API function -:c:func:`zfp_decompress`. - -Currently float arrays have a different compressed representation from -compressed double arrays due to differences in exponent width. It is not -possible to compress a double array and then decompress (demote) the result -to floats, for instance. Future versions of the |zfp| codec may use a unified -representation that does allow this. +A: Usually, but there are exceptions. When decompressing one block at a time +using the :ref:`low-level API `, it is possible to use more tightly +constrained :c:type:`zfp_stream` parameters during decompression than were +used during compression. For instance, one may use a smaller +:c:member:`zfp_stream.maxbits`, smaller :c:member:`zfp_stream.maxprec`, or +larger :c:member:`zfp_stream.minexp` during decompression to process fewer +compressed bits than are stored, and to decompress the array more quickly at +a lower precision. This may be useful in situations where the precision and +accuracy requirements are not known a priori, thus forcing conservative +settings during compression, or when the compressed stream is used for +multiple purposes. For instance, visualization usually has less stringent +precision requirements than quantitative data analysis. This feature of +decompressing to a lower precision is particularly useful when the stream is +stored progressively (see :ref:`Q13 `). + +Note, however, that when doing so, the caller must manually fast-forward +the stream (using :c:func:`stream_rseek`) to the beginning of the next block +before decompressing it, which may require extra bookkeeping. + +Also note that one may not use less constrained parameters during +decompression, e.g., one cannot ask for more than +:c:member:`zfp_stream.maxprec` bits of precision when decompressing. +Furthermore, the parameters must agree between compression and decompression +when calling the high-level API function :c:func:`zfp_decompress`. + +With regards to the :c:type:`zfp_field` struct passed to +:c:func:`zfp_compress` and :c:func:`zfp_decompress`, field dimensions must +match between compression and decompression, however strides need not match +(see :ref:`Q16 `). Additionally, the scalar type, +:c:type:`zfp_type`, must match. For example, float arrays currently have a +compressed representation different from compressed double arrays due to +differences in exponent width. It is not possible to compress a double array +and then decompress (demote) the result to floats, for instance. Future +versions of the |zfp| codec may use a unified representation that does allow +this. + +By default, compression parameters and array metadata are not stored in the +compressed stream, as often such information is recorded separately. +However, the user may optionally record both compression parameters and array +metadata in a header at the beginning of the compressed stream; see +:c:func:`zfp_write_header`, :c:func:`zfp_read_header`, and further discussion +:ref:`here `. ------------------------------------------------------------------------------- @@ -544,6 +561,33 @@ scalar fields can later be decompressed as non-interleaved fields:: using strides *sx* = 1, *sy* = *nx* and pointers :code:`&out[0][0][0]` and :code:`&out[1][0][0]`. +Another use case is when a compressed array is to be decompressed into a +larger surrounding array. For example, a 3D subarray with dimensions +*mx* |times| *my* |times| *mz* may be decompressed into a larger array +with dimensions *nx* |times| *ny* |times| *nz*, with *mx* |leq| *nx*, +*my* |leq| *ny*, *mz* |leq| *nz*. This can be achieved by setting the strides +to *sx* = 1, *sy* = *nx*, *sz* = *nx* |times| *ny* upon decompression (using +:c:func:`zfp_field_set_stride_3d`), while specifying *mx*, *my*, and *mz* as +the field dimensions (using :c:func:`zfp_field_3d` or +:c:func:`zfp_field_set_size_3d`). In this case, one may also wish to offset +the decompressed subarray to (*ox*, *oy*, *oz*) within the larger array +using:: + + float* data = new float[nx * ny * nz]; + pointer = data + ox * sx + oy * sy + oz * sz + +where *data* specifies the beginning of the larger array. *pointer* rather +than *data* would then be passed to :c:func:`zfp_field_3d` or +:c:func:`zfp_field_set_pointer` before calling :c:func:`zfp_decompress`. + +.. note:: + Strides are a property of the in-memory layout of an uncompressed array + and have no meaning with respect to the compressed bit stream representation + of the array. As such, |zfp| provides no mechanism for storing information + about the original strides in the compressed stream. If strides are to be + retained upon decompression, then the user needs to record them as auxiliary + metadata and initialize :c:type:`zfp_field` with them. + ------------------------------------------------------------------------------- .. _q-tolerance: diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 7819bb410..48ded1d4e 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -403,16 +403,26 @@ Types The strides, when nonzero, specify how the array is laid out in memory. Strides can be used in case multiple fields are stored interleaved via "array of struct" (AoS) rather than "struct of array" (SoA) storage, - or if the dimensions should be transposed during (de)compression. - Strides may even be negative, allowing one or more dimensions to be - traversed in reverse order. Given 4D array indices (*x*, *y*, *z*, *w*), - the corresponding array element is stored at - :: + or if the dimensions should be transposed during (de)compression; see + :ref:`this FAQ `. Strides may even be negative, allowing one + or more dimensions to be traversed in reverse order. Given 4D array + indices (*x*, *y*, *z*, *w*), the corresponding array element is stored + at:: data[x * sx + y * sy + z * sz + w * sw] where :code:`data` is a pointer to the first array element. +.. _field-match: +.. warning:: + The user must ensure that the scalar type and field dimensions *nx*, *ny*, + *nz*, and *nw* used during compression are matched during decompression. + Such metadata must either be maintained separately by the user or be + embedded in the compressed stream using :c:func:`zfp_write_header` and + recovered during decompression using :c:func:`zfp_read_header`. Strides, + on the other hand, need not match, which can be exploited, for example, to + decompress the array into a larger array. See :ref:`this FAQ ` and + the :ref:`tutorial ` for more details. .. _new-field: .. warning:: @@ -1022,6 +1032,14 @@ Compression and Decompression i.e., the current byte offset or the number of compressed bytes consumed. Zero is returned if decompression failed. + The field dimensions and scalar type used during decompression must match + those used during compression (see :c:type:`zfp_field`). This function + further requires that :c:type:`zfp_stream` is initialized with the same + :ref:`compression parameters ` used during compression. To assist + with this, an optional :ref:`header ` may be prepended that + encodes such metadata, which must be explicitly read using + :c:func:`zfp_read_header` to initialize *stream* and *field*. + ---- .. _zfp-header: @@ -1039,8 +1057,8 @@ Compression and Decompression .. c:function:: size_t zfp_read_header(zfp_stream* stream, zfp_field* field, uint mask) - Read header if one was previously written using :c:func:`zfp_write_header`. - The *stream* and *field* data structures are populated with the information + Read header previously written using :c:func:`zfp_write_header`. The + *stream* and *field* data structures are populated with the information stored in the header, as specified by the bit *mask* (see :c:macro:`macros `). The caller must ensure that *mask* agrees between header read and write calls. The return value is the number From 3b6eb55c775e8c22d89c2a584160e3fe81cf9f5a Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 22 Aug 2024 12:45:17 -0700 Subject: [PATCH 205/277] Add FAQ on zfp block size --- docs/source/faq.rst | 88 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 35908b600..a4db2d1b5 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -42,6 +42,7 @@ Questions answered in this FAQ: #. :ref:`How large a buffer is needed for compressed storage? ` #. :ref:`How can I print array values? ` #. :ref:`What is known about zfp compression errors? ` + #. :ref:`Why are zfp blocks 4 * 4 * 4 values? ` ------------------------------------------------------------------------------- @@ -1312,3 +1313,90 @@ done to combat such issues by supporting optional (left), errors are biased and depend on the relative location within a |zfp| block, resulting in errors not centered on zero. With proper rounding (right), errors are both smaller and unbiased. + +------------------------------------------------------------------------------- + +.. _q-block-size: + +Q31: *Why are zfp blocks 4 * 4 * 4 values?* + +One might ask why |zfp| uses *d*-dimensional blocks of |4powd| values and not +some other, perhaps configurable block size, *n*\ :sup:`d`. There are several +reasons why *n* = 4 was chosen: + +* For good performance, *n* should be an integer power of two so that indexing + can be done efficiently using bit masks and shifts rather than more + expensive division and modulo operations. As compression demands *n* > 1, + possible choices for *n* are 2, 4, 8, 16, ... + +* When *n* = 2, blocks are too small to exhibit significant redundancy; there + simply is insufficient spatial correlation to exploit for sufficient data + reduction. Additionally, excessive software cache thrashing would likely + occur for stencil computations, as even the smallest centered difference + stencil spans more than one block. Finally, per-block overhead in storage + (e.g., shared exponent, bit offset) and computation (e.g., software cache + lookup) could be amortized over only few values. Such small blocks were + immediately dismissed. + +* When *n* = 8, blocks are too large, for several reasons: + + * Each uncompressed block occupies a large number of hardware cache lines. + For example, a single 3D block of double-precision values would occupy + 4,096 bytes, which would represent a significant fraction of L1 cache. + |zfp| reduces data movement in computations by ensuring that repeated + accesses are to cached data rather than to main memory. + + * A generalization of the |zfp| :ref:`decorrelating transform ` + to *n* = 8 would require many more operations as well as "arbitrary" + numerical constants in need of expensive multiplication instead of cheap + bit shifts. The number of operations in this more general case scales as + *d* |times| *n*\ :sup:`d+1`. For *d* = 4, *n* = 8, this implies + 2\ :sup:`17` = 131,072 multiplications and 114,688 additions per block. + Contrast this with the algorithm optimized for *n* = 4, which uses only + 1,536 bit shifts and 2,560 additions or subtractions per 4D block. + + * The additional computational cost would also significantly increase the + latency of decoding a single block or filling a pipeline of concurrently + (de)compressed blocks, as in existing |zfp| hardware implementations. + + * The computational and cache storage overhead of accessing a single value + in a block would be very large: 4,096 values in *d* = 4 dimensions would + have to be decoded even if only one value were requested. + + * "Skinny" arrays would have to be padded to multiples of *n* = 8, which + could introduce an unaccepable storage overhead. For instance, a + 30 |times| 20 |times| 3 array of 1,800 values would be padded to + 32 |times| 24 |times| 8 = 6,144 values, an increase of about 3.4 times. + In contrast, when *n* = 4, only 32 |times| 20 |times| 4 = 2,560 values + would be needed, representing a 60% overhead. + + * The opportunity for data parallelism would be reduced by a factor of + 2\ :sup:`d` compared to using *n* = 4. The finer granularity and larger + number of blocks provided by *n* = 4 helps with load balancing and maps + well to today's GPUs that can concurrently process thousands of blocks. + + * With blocks comprising as many as 8\ :sup:`4` = 4,096 values, register + spillage would be substantial in GPU kernels for compression and + decompression. + +The choice *n* = 4 seems to be a sweet spot that well balances all of the +above factors. Additionally, *n* = 4 has these benefits: + + * *n* = 4 admits a very simple lifted implementation of the decorrelating + transform that can be performed using only integer addition, subtraction, + and bit shifts. + + * *n* = 4 allows taking advantage of AVX/SSE instructions designed for + vectors of length four, both in the (de)compression algorithm and + application code. + + * For 2D and 3D data, a block is 16 and 64 values, respectively, which + either equals or is close to the warp size on current GPU hardware. This + allows multiple cooperating threads to execute the same instruction on one + value in 1-4 blocks (either during (de)compression or in the numerical + application code). + + * Using a rate of 16 bits/value (a common choice for numerical computations), + a compressed 3D block occupies 128 bytes, or 1-2 hardware cache lines on + contemporary computers. Hence, a fair number of *compressed* blocks can + also fit in hardware cache. From 5ad8884e8faddf244c2c772897e0bc6bacfe5ab9 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 27 Aug 2024 09:11:10 -0700 Subject: [PATCH 206/277] Add gcc-12, include gcc-11 packages --- .github/workflows/tests.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index fce4abb37..46ce05435 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -27,6 +27,13 @@ jobs: - os: macos-latest cxx_compiler: g++-11 c_compiler: gcc-11 + packages: 'g++-11-multilib gcc-11-multilib' + omp: ON + target: all + + - os: macos-latest + cxx_compiler: g++-12 + c_compiler: gcc-12 omp: ON target: all From 432e5973674fb524138979116ca6289f80e5e612 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 27 Aug 2024 09:54:03 -0700 Subject: [PATCH 207/277] Discontinue gcc11, use omp with clang --- .github/workflows/tests.yml | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 46ce05435..0b554b68d 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -24,13 +24,6 @@ jobs: omp: ON target: all - - os: macos-latest - cxx_compiler: g++-11 - c_compiler: gcc-11 - packages: 'g++-11-multilib gcc-11-multilib' - omp: ON - target: all - - os: macos-latest cxx_compiler: g++-12 c_compiler: gcc-12 @@ -40,7 +33,7 @@ jobs: - os: macos-latest cxx_compiler: clang++ c_compiler: clang - omp: OFF + omp: ON target: all steps: From 3aa11f1c7987a093052eced4e666f611e5055d6d Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 27 Aug 2024 11:01:35 -0700 Subject: [PATCH 208/277] Revert omp with clang --- .github/workflows/tests.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 0b554b68d..9a4642853 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -33,7 +33,7 @@ jobs: - os: macos-latest cxx_compiler: clang++ c_compiler: clang - omp: ON + omp: OFF target: all steps: From 36b3fbba2f02173a631e148b1adf17beeee27b26 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 4 Sep 2024 11:08:15 -0700 Subject: [PATCH 209/277] Migrate quartz tests to dane --- tests/gitlab/dane-jobs.yml | 64 +++++++++++++++++++++++++++++++++ tests/gitlab/dane-templates.yml | 12 +++++++ tests/gitlab/gitlab-ci.yml | 6 ++-- 3 files changed, 80 insertions(+), 2 deletions(-) create mode 100644 tests/gitlab/dane-jobs.yml create mode 100644 tests/gitlab/dane-templates.yml diff --git a/tests/gitlab/dane-jobs.yml b/tests/gitlab/dane-jobs.yml new file mode 100644 index 000000000..b4405c2a3 --- /dev/null +++ b/tests/gitlab/dane-jobs.yml @@ -0,0 +1,64 @@ +########### +# CXX CPU # +########### + +cpp_gnu-10.3.1_build: + variables: + ci_cmake: "cmake/3.14.5" + ci_cxx_cmp: "g++" + ci_c_cmp: "gcc" + ci_cmp_mod: "gcc/10.3.1" + extends: [.cpp, .dane_build_cpu] + needs: [] + +cpp_gnu-10.3.1_test: + extends: [.dane_test_cpu] + needs: [cpp_gnu-10.3.1_build] + + +cpp_clang-14.0.6_build: + variables: + ci_cmake: "cmake/3.14.5" + ci_cxx_cmp: "clang++" + ci_c_cmp: "clang" + ci_cmp_mod: "clang/14.0.6" + extends: [.cpp, .dane_build_cpu] + needs: [] + +cpp_clang-14.0.6_test: + extends: [.dane_test_cpu] + needs: [cpp_clang-14.0.6_build] + + +cpp_intel-2022.1.0_build: + variables: + ci_cmake: "cmake/3.14.5" + ci_cxx_cmp: "icpc" + ci_c_cmp: "icc" + ci_cmp_mod: "intel/2022.1.0" + extends: [.cpp, .dane_build_cpu] + needs: [] + +cpp_intel-2022.1.0_test: + extends: [.dane_test_cpu] + needs: [cpp_intel-2022.1.0_build] + + + +######### +# C CPU # +######### + +c_gnu-10.3.1_build: + variables: + ci_cmake: "cmake/3.14.5" + ci_c_cmp: "gcc" + ci_cmp_mod: "gcc/10.3.1" + extends: [.c, .dane_build_cpu] + needs: [] + +c_gnu-10.3.1_test: + variables: + ci_test_regex: "Cfp" + extends: [.dane_test_cpu] + needs: [c_gnu-10.3.1_build] diff --git a/tests/gitlab/dane-templates.yml b/tests/gitlab/dane-templates.yml new file mode 100644 index 000000000..147beb33b --- /dev/null +++ b/tests/gitlab/dane-templates.yml @@ -0,0 +1,12 @@ +.dane_job: + tags: + - batch + - dane + +.dane_build_cpu: + extends: [.build_cpu, .dane_job] + +.dane_test_cpu: + variables: + ci_test_regex: "." + extends: [.test_cpu, .dane_job] diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 8a4c5a85b..665855736 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -139,7 +139,9 @@ include: file: 'id_tokens.yml' - local: tests/gitlab/pascal-templates.yml - local: tests/gitlab/pascal-jobs.yml - - local: tests/gitlab/quartz-templates.yml - - local: tests/gitlab/quartz-jobs.yml + - local: tests/gitlab/dane-templates.yml + - local: tests/gitlab/dane-jobs.yml +# - local: tests/gitlab/quartz-templates.yml +# - local: tests/gitlab/quartz-jobs.yml # - local: tests/gitlab/corona-templates.yml # - local: tests/gitlab/corona-jobs.yml From b4c4696a0e1522a6e60e4a3b701b4a83394fb6fc Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 17 Sep 2024 19:10:40 -0700 Subject: [PATCH 210/277] Avoid integer overflow in testzfp --- tests/testzfp.cpp | 80 +++++++++++++++++++++++------------------------ 1 file changed, 40 insertions(+), 40 deletions(-) diff --git a/tests/testzfp.cpp b/tests/testzfp.cpp index 82b4074ec..9a547c487 100644 --- a/tests/testzfp.cpp +++ b/tests/testzfp.cpp @@ -44,16 +44,16 @@ test_size(ArraySize size) inline void refine1d(int* g, const int* f, size_t m) { - const int weight[4] = { -1, 9, 9, -1 }; + const int64 weight[4] = { -1, 9, 9, -1 }; const size_t n = 2 * m; for (size_t x = 0; x < n; x++) { - int s = 0; + int64 s = 0; for (size_t i = 0; i < 4; i++) { size_t xx = x & 1u ? (x / 2 + i - 1 + m) % m : x / 2; s += weight[i] * f[xx]; } - g[x] = s / 16; + g[x] = static_cast(s / 16); } } @@ -61,12 +61,12 @@ refine1d(int* g, const int* f, size_t m) inline void refine2d(int* g, const int* f, size_t m) { - const int weight[4] = { -1, 9, 9, -1 }; + const int64 weight[4] = { -1, 9, 9, -1 }; const size_t n = 2 * m; for (size_t y = 0; y < n; y++) for (size_t x = 0; x < n; x++) { - int s = 0; + int64 s = 0; for (size_t j = 0; j < 4; j++) { size_t yy = y & 1u ? (y / 2 + j - 1 + m) % m : y / 2; for (size_t i = 0; i < 4; i++) { @@ -74,7 +74,7 @@ refine2d(int* g, const int* f, size_t m) s += weight[i] * weight[j] * f[xx + m * yy]; } } - g[x + n * y] = s / (16 * 16); + g[x + n * y] = static_cast(s / (16 * 16)); } } @@ -82,13 +82,13 @@ refine2d(int* g, const int* f, size_t m) inline void refine3d(int* g, const int* f, size_t m) { - const int weight[4] = { -1, 9, 9, -1 }; + const int64 weight[4] = { -1, 9, 9, -1 }; const size_t n = 2 * m; for (size_t z = 0; z < n; z++) for (size_t y = 0; y < n; y++) for (size_t x = 0; x < n; x++) { - int s = 0; + int64 s = 0; for (size_t k = 0; k < 4; k++) { size_t zz = z & 1u ? (z / 2 + k - 1 + m) % m : z / 2; for (size_t j = 0; j < 4; j++) { @@ -99,7 +99,7 @@ refine3d(int* g, const int* f, size_t m) } } } - g[x + n * (y + n * z)] = s / (16 * 16 * 16); + g[x + n * (y + n * z)] = static_cast(s / (16 * 16 * 16)); } } @@ -107,14 +107,14 @@ refine3d(int* g, const int* f, size_t m) inline void refine4d(int* g, const int* f, size_t m) { - const int weight[4] = { -1, 9, 9, -1 }; + const int64 weight[4] = { -1, 9, 9, -1 }; const size_t n = 2 * m; for (size_t w = 0; w < n; w++) for (size_t z = 0; z < n; z++) for (size_t y = 0; y < n; y++) for (size_t x = 0; x < n; x++) { - int s = 0; + int64 s = 0; for (size_t l = 0; l < 4; l++) { size_t ww = w & 1u ? (w / 2 + l - 1 + m) % m : w / 2; for (size_t k = 0; k < 4; k++) { @@ -128,7 +128,7 @@ refine4d(int* g, const int* f, size_t m) } } } - g[x + n * (y + n * (z + n * w))] = s / (16 * 16 * 16 * 16); + g[x + n * (y + n * (z + n * w))] = static_cast(s / (16 * 16 * 16 * 16)); } } @@ -792,11 +792,11 @@ test(uint dims, ArraySize array_size) // test data integrity uint32 checksum[2][2][4] = { // [size][type][dims] // small - {{ 0x54174c44u, 0x86609589u, 0xfc0a6a76u, 0xa3481e00u }, - { 0x7d257bb6u, 0x294bb210u, 0x68614d26u, 0xf6bd3a21u }}, + {{ 0x54174c44u, 0x86609589u, 0xfc0a6a76u, 0x28708a2bu }, + { 0x7d257bb6u, 0x294bb210u, 0x68614d26u, 0xd58a5fe7u }}, // large - {{ 0xd1ce1aceu, 0x644274dau, 0xc0ad63fau, 0x700de480u }, - { 0xc3ed7116u, 0x644e2117u, 0xd7464b07u, 0x2516382eu }}, + {{ 0xd1ce1aceu, 0x644274dau, 0xc0ad63fau, 0xdc65b02eu }, + { 0xc3ed7116u, 0x644e2117u, 0xd7464b07u, 0xe4b60fbbu }}, }; uint32 h = hash(f, n * sizeof(Scalar)); if (h != checksum[array_size][t][dims - 1]) @@ -815,13 +815,13 @@ test(uint dims, ArraySize array_size) {1.627e+01, 8.277e-02, 0.000e+00}, {1.500e+00, 3.663e-03, 0.000e+00}, {1.500e+00, 9.583e-03, 0.000e+00}, - {1.373e+01, 6.633e-01, 0.000e+00}, + {6.750e+00, 1.931e-01, 0.000e+00}, }, { {1.627e+01, 1.601e+01, 1.832e-04, 0.000e+00}, {2.376e+01, 1.797e-01, 8.584e-06, 0.000e+00}, {5.210e+00, 2.002e-01, 3.338e-05, 0.000e+00}, - {1.016e+01, 8.985e+00, 3.312e-03, 0.000e+00}, + {9.594e+00, 2.264e+00, 8.282e-04, 0.000e+00}, }, }, // large @@ -830,13 +830,13 @@ test(uint dims, ArraySize array_size) {1.627e+01, 2.100e-02, 0.000e+00}, {1.624e-01, 7.439e-05, 0.000e+00}, {1.001e-02, 7.248e-05, 0.000e+00}, - {2.527e-02, 2.460e-04, 0.000e+00}, + {1.038e-02, 1.078e-04, 0.000e+00}, }, { {1.627e+01, 1.601e+01, 2.289e-05, 0.000e+00}, {1.607e+01, 2.076e-03, 0.000e+00, 0.000e+00}, {1.407e-01, 7.344e-04, 0.000e+00, 0.000e+00}, - {1.436e-01, 2.659e-03, 8.801e-08, 0.000e+00}, + {8.130e-02, 1.515e-03, 4.401e-08, 0.000e+00}, } } }; @@ -856,13 +856,13 @@ test(uint dims, ArraySize array_size) {2192, 3280, 6328}, { 592, 1328, 4384}, { 152, 1040, 4600}, - { 64, 1760, 5856}, + { 32, 352, 4168}, }, { {3664, 6712, 14104}, {1424, 4480, 12616}, {1064, 4624, 12808}, - {1768, 5864, 14056}, + { 360, 4168, 12360}, }, }, // large @@ -871,13 +871,13 @@ test(uint dims, ArraySize array_size) {8965672, 13160560, 21835352}, {2235560, 3512848, 10309240}, { 568456, 1361056, 8759696}, - { 134344, 739632, 8896360}, + { 135344, 706600, 8207768}, }, { {14733112, 23407904, 44997832}, { 3905240, 10701640, 40856544}, { 1458368, 8857008, 41270184}, - { 763928, 8920656, 41574712}, + { 730896, 8232056, 40581448}, }, } }; @@ -895,13 +895,13 @@ test(uint dims, ArraySize array_size) {6328, 11944, 13720}, {4936, 11064, 12520}, {6104, 11752, 12784}, - {9440, 14048, 14048}, + {8776, 12360, 12360}, }, { {6712, 25888, 29064}, {5032, 26016, 28984}, {6128, 27120, 29192}, - {9448, 30440, 30440}, + {8776, 28744, 28744}, }, }, // large @@ -910,13 +910,13 @@ test(uint dims, ArraySize array_size) {21815976, 38285256, 43425280}, { 9187232, 32695984, 40464144}, { 8914336, 33364208, 41172864}, - {12109200, 35921784, 41550416}, + {11394368, 34992872, 40557152}, }, { {23388528, 79426016, 88659304}, { 9579632, 89770896, 103388072}, { 9011648, 94009072, 107606336}, - {12133496, 97126288, 107911568}, + {11418664, 96325984, 106922328}, }, } }; @@ -933,13 +933,13 @@ test(uint dims, ArraySize array_size) 7272, 5104, 6096, - 6864, + 7208, }, { 7784, 5232, 6128, - 6872, + 7216, }, }, // large @@ -948,13 +948,13 @@ test(uint dims, ArraySize array_size) 25037288, 12792440, 14187128, - 17135704, + 17222720, }, { 27134024, 13315632, 14316880, - 17168096, + 17255112, }, } }; @@ -965,25 +965,25 @@ test(uint dims, ArraySize array_size) double emax[2][2][4] = { // [size][type][dims] (construct test) // small { - {4.578e-05, 7.630e-06, 3.148e-05, 3.598e-03}, - {1.832e-04, 8.584e-06, 3.338e-05, 3.312e-03}, + {4.578e-05, 7.630e-06, 3.148e-05, 8.197e-04}, + {1.832e-04, 8.584e-06, 3.338e-05, 8.282e-04}, }, // large { - {0.000e+00, 0.000e+00, 0.000e+00, 1.193e-07}, - {2.289e-05, 0.000e+00, 0.000e+00, 8.801e-08}, + {0.000e+00, 0.000e+00, 0.000e+00, 2.981e-08}, + {2.289e-05, 0.000e+00, 0.000e+00, 4.401e-08}, } }; double dfmax[2][2][4] = { // [size][type][dims] (update test) // small { - {2.155e-02, 3.755e-01, 1.846e+00, 4.843e+01}, - {2.155e-02, 3.755e-01, 1.846e+00, 4.844e+01}, + {2.155e-02, 3.755e-01, 1.846e+00, 1.601e+01}, + {2.155e-02, 3.755e-01, 1.846e+00, 1.601e+01}, }, // large { - {2.441e-04, 4.883e-04, 1.222e-03, 2.567e-02}, - {2.670e-04, 4.883e-04, 1.222e-03, 2.567e-02}, + {2.441e-04, 4.883e-04, 1.222e-03, 8.794e-03}, + {2.670e-04, 4.883e-04, 1.222e-03, 8.795e-03}, } }; double rate = 16; From 3eafd7c9b6cec30e55de276bdd9fc9b75d110492 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 17 Sep 2024 19:24:11 -0700 Subject: [PATCH 211/277] Avoid integer undefined behavior (resolves #241) --- CHANGELOG.md | 8 ++++++++ src/template/decode.c | 19 +++++++++++++++---- src/template/revdecode.c | 13 ++++++++----- src/template/revencode.c | 13 ++++++++----- 4 files changed, 39 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 96aa92169..d8ca4459f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,14 @@ Change Log --- +## Unreleased + +### Fixed + +- #241: Signed left shifts, integer overflow invoke undefined behavior. + +--- + ## 1.0.1 (2023-12-15) This patch release primarily addresses minor bug fixes and is needed to update diff --git a/src/template/decode.c b/src/template/decode.c index 2f39bebd5..aff87c4d2 100644 --- a/src/template/decode.c +++ b/src/template/decode.c @@ -16,16 +16,27 @@ _t1(inv_lift, Int)(Int* p, ptrdiff_t s) /* ** non-orthogonal transform + ** ** ( 4 6 -4 -1) (x) ** 1/4 * ( 4 2 4 5) (y) ** ( 4 -2 4 -5) (z) ** ( 4 -6 -4 1) (w) + ** + ** original lifted version, which invokes UB due to signed left shift and + ** integer overflow: + ** + ** y += w >> 1; w -= y >> 1; + ** y += w; w <<= 1; w -= y; + ** z += x; x <<= 1; x -= z; + ** y += z; z <<= 1; z -= y; + ** w += x; x <<= 1; x -= w; */ + y += w >> 1; w -= y >> 1; - y += w; w <<= 1; w -= y; - z += x; x <<= 1; x -= z; - y += z; z <<= 1; z -= y; - w += x; x <<= 1; x -= w; + y += w; w -= y - w; + z += x; x -= z - x; + y += z; z -= y - z; + w += x; x -= w - x; p -= s; *p = w; p -= s; *p = z; diff --git a/src/template/revdecode.c b/src/template/revdecode.c index 30ec1f76a..655e1ae5c 100644 --- a/src/template/revdecode.c +++ b/src/template/revdecode.c @@ -6,7 +6,7 @@ static void _t2(rev_inv_xform, Int, DIMS)(Int* p); static void _t1(rev_inv_lift, Int)(Int* p, ptrdiff_t s) { - Int x, y, z, w; + UInt x, y, z, w; x = *p; p += s; y = *p; p += s; z = *p; p += s; @@ -14,19 +14,22 @@ _t1(rev_inv_lift, Int)(Int* p, ptrdiff_t s) /* ** high-order Lorenzo transform (P4 Pascal matrix) + ** ** ( 1 0 0 0) (x) ** ( 1 1 0 0) (y) ** ( 1 2 1 0) (z) ** ( 1 3 3 1) (w) + ** + ** unsigned arithmetic is used to avoid integer overflow */ w += z; z += y; w += z; y += x; z += y; w += z; - p -= s; *p = w; - p -= s; *p = z; - p -= s; *p = y; - p -= s; *p = x; + p -= s; *p = (Int)w; + p -= s; *p = (Int)z; + p -= s; *p = (Int)y; + p -= s; *p = (Int)x; } /* decode block of integers using reversible algorithm */ diff --git a/src/template/revencode.c b/src/template/revencode.c index f68df269a..9c4938d74 100644 --- a/src/template/revencode.c +++ b/src/template/revencode.c @@ -6,7 +6,7 @@ static void _t2(rev_fwd_xform, Int, DIMS)(Int* p); static void _t1(rev_fwd_lift, Int)(Int* p, ptrdiff_t s) { - Int x, y, z, w; + UInt x, y, z, w; x = *p; p += s; y = *p; p += s; z = *p; p += s; @@ -14,19 +14,22 @@ _t1(rev_fwd_lift, Int)(Int* p, ptrdiff_t s) /* ** high-order Lorenzo transform + ** ** ( 1 0 0 0) (x) ** (-1 1 0 0) (y) ** ( 1 -2 1 0) (z) ** (-1 3 -3 1) (w) + ** + ** unsigned arithmetic is used to avoid integer overflow */ w -= z; z -= y; y -= x; w -= z; z -= y; w -= z; - p -= s; *p = w; - p -= s; *p = z; - p -= s; *p = y; - p -= s; *p = x; + p -= s; *p = (Int)w; + p -= s; *p = (Int)z; + p -= s; *p = (Int)y; + p -= s; *p = (Int)x; } /* return precision required to encode block reversibly */ From 212a24ed21857af66123f7f94387fa977e308c33 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 18 Sep 2024 10:10:04 -0700 Subject: [PATCH 212/277] Fix integer overflow in CUDA decoder --- src/cuda_zfp/shared.h | 57 ++++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git a/src/cuda_zfp/shared.h b/src/cuda_zfp/shared.h index 27df25beb..547e85f5e 100644 --- a/src/cuda_zfp/shared.h +++ b/src/cuda_zfp/shared.h @@ -223,29 +223,40 @@ __device__ static void inv_lift(Int* p) { - Int x, y, z, w; - x = *p; p += s; - y = *p; p += s; - z = *p; p += s; - w = *p; p += s; - - /* - ** non-orthogonal transform - ** ( 4 6 -4 -1) (x) - ** 1/4 * ( 4 2 4 5) (y) - ** ( 4 -2 4 -5) (z) - ** ( 4 -6 -4 1) (w) - */ - y += w >> 1; w -= y >> 1; - y += w; w <<= 1; w -= y; - z += x; x <<= 1; x -= z; - y += z; z <<= 1; z -= y; - w += x; x <<= 1; x -= w; - - p -= s; *p = w; - p -= s; *p = z; - p -= s; *p = y; - p -= s; *p = x; + Int x, y, z, w; + x = *p; p += s; + y = *p; p += s; + z = *p; p += s; + w = *p; p += s; + + /* + ** non-orthogonal transform + ** + ** ( 4 6 -4 -1) (x) + ** 1/4 * ( 4 2 4 5) (y) + ** ( 4 -2 4 -5) (z) + ** ( 4 -6 -4 1) (w) + ** + ** original lifted version, which invokes UB due to signed left shift and + ** integer overflow: + ** + ** y += w >> 1; w -= y >> 1; + ** y += w; w <<= 1; w -= y; + ** z += x; x <<= 1; x -= z; + ** y += z; z <<= 1; z -= y; + ** w += x; x <<= 1; x -= w; + */ + + y += w >> 1; w -= y >> 1; + y += w; w -= y - w; + z += x; x -= z - x; + y += z; z -= y - z; + w += x; x -= w - x; + + p -= s; *p = w; + p -= s; *p = z; + p -= s; *p = y; + p -= s; *p = x; } From b621b47d072935d0dbc11674b903467a634c7aa0 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 18 Sep 2024 12:06:11 -0700 Subject: [PATCH 213/277] Filter testviews output --- tests/testviews.cpp | 69 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 58 insertions(+), 11 deletions(-) diff --git a/tests/testviews.cpp b/tests/testviews.cpp index bc3d4b3b6..f6f09f3c0 100644 --- a/tests/testviews.cpp +++ b/tests/testviews.cpp @@ -30,6 +30,31 @@ verify(double f, double g) } } +// filter output; returns true for first head and last tail calls +static bool +filter_output(size_t head = 0, size_t tail = 0, size_t size = 0) +{ + static size_t i = 0; + static size_t h = 0; + static size_t t = 0; + static size_t n = 0; + + if (size) { + i = 0; + h = head; + t = tail; + n = size; + return false; + } + + bool display = !(h <= i && i + t < n); + if (!display && i == h) + std::cout << "..." << std::endl; + i++; + + return display; +} + static int usage() { @@ -98,30 +123,36 @@ int main(int argc, char* argv[]) // rectangular view into a std::cout << std::endl << "3D view" << std::endl; zfp::array3::view v(&a, x0, y0, z0, mx, my, mz); + filter_output(v.size_x() + 2, 3, v.size()); for (size_t z = 0; z < v.size_z(); z++) for (size_t y = 0; y < v.size_y(); y++) for (size_t x = 0; x < v.size_x(); x++) { - std::cout << x << " " << y << " " << z << ": " << a(x0 + x, y0 + y, z0 + z) << " " << v(x, y, z) << std::endl; + if (filter_output()) + std::cout << x << " " << y << " " << z << ": " << a(x0 + x, y0 + y, z0 + z) << " " << v(x, y, z) << std::endl; verify(a(x0 + x, y0 + y, z0 + z), v(x, y, z)); } // flat view of all of a std::cout << std::endl << "3D flat view" << std::endl; zfp::array3::flat_view fv(&a); + filter_output(fv.size_x() + 2, 3, fv.size()); for (size_t z = 0; z < fv.size_z(); z++) for (size_t y = 0; y < fv.size_y(); y++) for (size_t x = 0; x < fv.size_x(); x++) { - std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << fv[fv.index(x, y, z)] << std::endl; + if (filter_output()) + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << fv[fv.index(x, y, z)] << std::endl; verify(a(x, y, z), fv[fv.index(x, y, z)]); } // nested view of all of a std::cout << std::endl << "3D nested view" << std::endl; zfp::array3::nested_view nv(&a); + filter_output(nv.size_x() + 2, 3, nv.size()); for (size_t z = 0; z < nv.size_z(); z++) for (size_t y = 0; y < nv.size_y(); y++) for (size_t x = 0; x < nv.size_x(); x++) { - std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << nv[z][y][x] << std::endl; + if (filter_output()) + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << nv[z][y][x] << std::endl; verify(a(x, y, z), nv[z][y][x]); } @@ -130,12 +161,13 @@ int main(int argc, char* argv[]) zfp::array3::view::const_reference vr = v(0, 0, 0); zfp::array3::view::const_pointer p = &vr; p = &v(0, 0, 0); + filter_output(v.size_x() + 2, 3, v.size()); for (zfp::array3::view::const_iterator it = v.begin(); it != v.end(); it++) { size_t x = it.i(); size_t y = it.j(); size_t z = it.k(); -std::cout << x << " " << y << " " << z << std::endl; -std::cout << mx << " " << my << " " << std::endl; + if (filter_output()) + std::cout << x << " " << y << " " << z << ": " << *it << " " << p[x + mx * (y + my * z)] << std::endl; verify(*it, p[x + mx * (y + my * z)]); } @@ -144,10 +176,13 @@ std::cout << mx << " " << my << " " << std::endl; zfp::array3::flat_view::const_reference fvr = fv[0]; zfp::array3::flat_view::const_pointer fp = &fvr; fp = &fv(0, 0, 0); + filter_output(fv.size_x() + 2, 3, fv.size()); for (zfp::array3::flat_view::const_iterator it = fv.begin(); it != fv.end(); it++) { size_t x = it.i(); size_t y = it.j(); size_t z = it.k(); + if (filter_output()) + std::cout << x << " " << y << " " << z << ": " << *it << " " << fp[x + nx * (y + ny * z)] << std::endl; verify(*it, fp[x + nx * (y + ny * z)]); } @@ -155,18 +190,22 @@ std::cout << mx << " " << my << " " << std::endl; std::cout << std::endl << "2D slice" << std::endl; size_t z = rand(0, nv.size_z() - 1); zfp::array3::nested_view2 slice2(nv[z]); + filter_output(slice2.size_x() + 2, 3, slice2.size()); for (size_t y = 0; y < slice2.size_y(); y++) for (size_t x = 0; x < slice2.size_x(); x++) { - std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << slice2[y][x] << std::endl; + if (filter_output()) + std::cout << x << " " << y << " " << z << ": " << a(x, y, z) << " " << slice2[y][x] << std::endl; verify(a(x, y, z), slice2[y][x]); } // 2D array constructed from 2D slice (exercises deep copy via iterator) std::cout << std::endl << "2D array from 2D slice" << std::endl; zfp::array2 b(slice2); + filter_output(b.size_x() + 2, 3, b.size()); for (size_t y = 0; y < b.size_y(); y++) for (size_t x = 0; x < b.size_x(); x++) { - std::cout << x << " " << y << ": " << b(x, y) << " " << slice2[y][x] << std::endl; + if (filter_output()) + std::cout << x << " " << y << ": " << b(x, y) << " " << slice2[y][x] << std::endl; verify(b(x, y), slice2[y][x]); } @@ -182,18 +221,22 @@ std::cout << mx << " " << my << " " << std::endl; // 2D array constructed from 2D slice of 3D array (exercises deep copy via iterator) std::cout << std::endl << "2D array from 2D slice of 3D array" << std::endl; zfp::array2 c(slice2); + filter_output(c.size_x() + 2, 3, c.size()); for (size_t y = 0; y < c.size_y(); y++) for (size_t x = 0; x < c.size_x(); x++) { - std::cout << x << " " << y << ": " << c(x, y) << " " << slice2[y][x] << std::endl; + if (filter_output()) + std::cout << x << " " << y << ": " << c(x, y) << " " << slice2[y][x] << std::endl; verify(c(x, y), slice2[y][x]); } // 2D thread-safe read-only view of c std::cout << std::endl << "2D private read-only view" << std::endl; zfp::array2::private_const_view d(&c); + filter_output(c.size_x() + 2, 3, c.size()); for (size_t y = 0; y < c.size_y(); y++) for (size_t x = 0; x < c.size_x(); x++) { - std::cout << x << " " << y << ": " << c(x, y) << " " << d(x, y) << std::endl; + if (filter_output()) + std::cout << x << " " << y << ": " << c(x, y) << " " << d(x, y) << std::endl; verify(c(x, y), d(x, y)); } @@ -206,10 +249,12 @@ std::cout << mx << " " << my << " " << std::endl; { // make a thread-local view into c zfp::array2::private_const_view d(&c); + if (omp_get_thread_num() == 0) + filter_output(d.size_x() + 2, 3, d.size()); for (size_t y = 0; y < d.size_y(); y++) for (size_t x = 0; x < d.size_x(); x++) { double val = data[x + nx * y]; - if (omp_get_thread_num() == 0) + if (omp_get_thread_num() == 0 && filter_output()) std::cout << x << " " << y << ": " << val << " " << d(x, y) << std::endl; verify(val, d(x, y)); } @@ -221,13 +266,15 @@ std::cout << mx << " " << my << " " << std::endl; // partition c into disjoint views zfp::array2::private_view d(&c); d.partition(omp_get_thread_num(), omp_get_num_threads()); + if (omp_get_thread_num() == 0) + filter_output(d.size_x() + 2, 3, d.size()); for (size_t j = 0; j < d.size_y(); j++) for (size_t i = 0; i < d.size_x(); i++) { d(i, j) += 1; size_t x = d.global_x(i); size_t y = d.global_y(j); double val = data[x + nx * y] + 1; - if (omp_get_thread_num() == 0) + if (omp_get_thread_num() == 0 && filter_output()) std::cout << x << " " << y << ": " << val << " " << d(i, j) << std::endl; verify(val, d(i, j)); } From 632a128638a110231a835ce434d678b9963cf23f Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 19 Sep 2024 08:23:06 -0700 Subject: [PATCH 214/277] Fix GitLab SLURM parameters --- tests/gitlab/gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 665855736..ec357893a 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -4,7 +4,7 @@ variables: GIT_SUBMODULE_STRATEGY: recursive - LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -A asccasc -t 00:20:00" + LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -t 00:20:00" LLNL_SERVICE_USER: zfp stages: From a889bdfeac7f172a244ad9eb957dd9ae05214a0e Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 16 Oct 2024 14:06:07 -0700 Subject: [PATCH 215/277] Migrate CUDA tests from Pascal to Lassen --- tests/gitlab/gitlab-ci.yml | 6 ++++-- tests/gitlab/lassen-jobs.yml | 17 +++++++++++++++++ tests/gitlab/lassen-templates.yml | 12 ++++++++++++ 3 files changed, 33 insertions(+), 2 deletions(-) create mode 100644 tests/gitlab/lassen-jobs.yml create mode 100644 tests/gitlab/lassen-templates.yml diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index ec357893a..8fabca5fd 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -137,8 +137,10 @@ stages: include: - project: 'lc-templates/id_tokens' file: 'id_tokens.yml' - - local: tests/gitlab/pascal-templates.yml - - local: tests/gitlab/pascal-jobs.yml +# - local: tests/gitlab/pascal-templates.yml +# - local: tests/gitlab/pascal-jobs.yml + - local: tests/gitlab/lassen-templates.yml + - local: tests/gitlab/lassen-jobs.yml - local: tests/gitlab/dane-templates.yml - local: tests/gitlab/dane-jobs.yml # - local: tests/gitlab/quartz-templates.yml diff --git a/tests/gitlab/lassen-jobs.yml b/tests/gitlab/lassen-jobs.yml new file mode 100644 index 000000000..ef76a21e6 --- /dev/null +++ b/tests/gitlab/lassen-jobs.yml @@ -0,0 +1,17 @@ +############ +# CUDA GPU # +############ + +cuda-11.6.1_build: + variables: + ci_cmake: "cmake/3.14.5" + ci_cmp_mod: "cuda/11.6.1" + ci_gcc_mod: "gcc/8.3.1" + extends: [.cuda, .lassen_build_gpu] + needs: [] + +cuda-11.6.1_test: + variables: + ci_test_regex: "Cuda" + extends: [.pascal_test_gpu] + needs: [cuda-11.8.0_build] diff --git a/tests/gitlab/lassen-templates.yml b/tests/gitlab/lassen-templates.yml new file mode 100644 index 000000000..c636ba41e --- /dev/null +++ b/tests/gitlab/lassen-templates.yml @@ -0,0 +1,12 @@ +.lassen_job: + tags: + - batch + - lassen + +.lassen_build_gpu: + extends: [.build_gpu, .lassen_job] + +.lassen_test_gpu: + variables: + ci_test_regex: "." + extends: [.test_gpu, .lassen_job] From ad3f420baf332d6d2bd7b30a511928c681622afd Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 16 Oct 2024 14:30:56 -0700 Subject: [PATCH 216/277] Fix missing machine name substitution --- tests/gitlab/lassen-jobs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/gitlab/lassen-jobs.yml b/tests/gitlab/lassen-jobs.yml index ef76a21e6..4959e9a09 100644 --- a/tests/gitlab/lassen-jobs.yml +++ b/tests/gitlab/lassen-jobs.yml @@ -13,5 +13,5 @@ cuda-11.6.1_build: cuda-11.6.1_test: variables: ci_test_regex: "Cuda" - extends: [.pascal_test_gpu] + extends: [.lassen_test_gpu] needs: [cuda-11.8.0_build] From da9a8349ab200890b169b90ecde2b8a6a6fc6de8 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 16 Oct 2024 14:58:12 -0700 Subject: [PATCH 217/277] Correct CUDA version for Lassen tests --- tests/gitlab/lassen-jobs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/gitlab/lassen-jobs.yml b/tests/gitlab/lassen-jobs.yml index 4959e9a09..71951ea02 100644 --- a/tests/gitlab/lassen-jobs.yml +++ b/tests/gitlab/lassen-jobs.yml @@ -14,4 +14,4 @@ cuda-11.6.1_test: variables: ci_test_regex: "Cuda" extends: [.lassen_test_gpu] - needs: [cuda-11.8.0_build] + needs: [cuda-11.6.1_build] From 8c0a9cde93b740ace708130eb8fdd22ea0469c8e Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 27 Oct 2024 10:46:08 -0700 Subject: [PATCH 218/277] Add discussion of and references to finite-difference errors --- docs/source/faq.rst | 60 ++++++++++++++++++++++++++++----------------- 1 file changed, 37 insertions(+), 23 deletions(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index a4db2d1b5..9c82a2f0e 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -1275,18 +1275,24 @@ Q30: *What is known about zfp compression errors?* A: Significant effort has been spent on characterizing compression errors resulting from |zfp|, as detailed in the following publications: -* P. Lindstrom, - "`Error Distributions of Lossy Floating-Point Compressors `__," - JSM 2017 Proceedings. -* J. Diffenderfer, A. Fox, J. Hittinger, G. Sanders, P. Lindstrom, - "`Error Analysis of ZFP Compression for Floating-Point Data `__," - SIAM Journal on Scientific Computing, 2019. -* D. Hammerling, A. Baker, A. Pinard, P. Lindstrom, - "`A Collaborative Effort to Improve Lossy Compression Methods for Climate Data `__," - 5th International Workshop on Data Analysis and Reduction for Big Scientific Data, 2019. -* A. Fox, J. Diffenderfer, J. Hittinger, G. Sanders, P. Lindstrom. - "`Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods `__," - SIAM Journal on Scientific Computing, 2020. +#. P. Lindstrom, + "`Error Distributions of Lossy Floating-Point Compressors `__," + JSM 2017 Proceedings. +#. J. Diffenderfer, A. Fox, J. Hittinger, G. Sanders, P. Lindstrom, + "`Error Analysis of ZFP Compression for Floating-Point Data `__," + SIAM Journal on Scientific Computing, 2019. +#. D. Hammerling, A. Baker, A. Pinard, P. Lindstrom, + "`A Collaborative Effort to Improve Lossy Compression Methods for Climate Data `__," + 5th International Workshop on Data Analysis and Reduction for Big Scientific Data, 2019. +#. A. Fox, J. Diffenderfer, J. Hittinger, G. Sanders, P. Lindstrom. + "`Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods `__," + SIAM Journal on Scientific Computing, 2020. +#. P. Lindstrom, J. Hittinger, J. Diffenderfer, A. Fox, D. Osei-Kuffuor, J. Banks. + "`ZFP: A Compressed Array Representation for Numerical Computations `__," + International Journal of High-Performance Computing Applications, 2024. +#. A. Fox, P. Lindstrom. + "`Statistical Analysis of ZFP: Understanding Bias `__," + LLNL-JRNL-858256, Lawrence Livermore National Laboratory, 2024. In short, |zfp| compression errors are roughly normally distributed as a consequence of the central limit theorem, and can be bounded. Because the @@ -1297,9 +1303,8 @@ are far smaller than the absolute error tolerance specified in (see :ref:`Q22 `). It is known that |zfp| errors can be slightly biased and correlated (see -:numref:`zfp-rounding` and the third paper above). Recent work has been -done to combat such issues by supporting optional -:ref:`rounding modes `. +:numref:`zfp-rounding` and papers #3 and #6 above). Recent work has been done +to combat such issues by supporting optional :ref:`rounding modes `. .. _zfp-rounding: .. figure:: zfp-rounding.pdf @@ -1314,11 +1319,20 @@ done to combat such issues by supporting optional block, resulting in errors not centered on zero. With proper rounding (right), errors are both smaller and unbiased. +It is also known how |zfp| compression errors behave as a function of grid +spacing, *h*. In particular, regardless of dimensionality, the compression +error *decreases* with finer grids (smaller *h*) for a given rate (i.e., +fixed compressed storage size). The |zfp| compression error decay is fast +enough that the corresponding error in partial derivative estimates based on +finite differences, which *increases* with smaller *h* when using conventional +floating point, instead *decreases* with finer grids when using |zfp|. See +paper #5 for details. + ------------------------------------------------------------------------------- .. _q-block-size: -Q31: *Why are zfp blocks 4 * 4 * 4 values?* +Q31: *Why are zfp blocks 4* |times| *4* |times| *4 values?* One might ask why |zfp| uses *d*-dimensional blocks of |4powd| values and not some other, perhaps configurable block size, *n*\ :sup:`d`. There are several @@ -1326,17 +1340,17 @@ reasons why *n* = 4 was chosen: * For good performance, *n* should be an integer power of two so that indexing can be done efficiently using bit masks and shifts rather than more - expensive division and modulo operations. As compression demands *n* > 1, - possible choices for *n* are 2, 4, 8, 16, ... + expensive division and modulo operations. As nontrivial compression demands + *n* > 1, possible choices for *n* are 2, 4, 8, 16, ... * When *n* = 2, blocks are too small to exhibit significant redundancy; there simply is insufficient spatial correlation to exploit for sufficient data reduction. Additionally, excessive software cache thrashing would likely occur for stencil computations, as even the smallest centered difference - stencil spans more than one block. Finally, per-block overhead in storage - (e.g., shared exponent, bit offset) and computation (e.g., software cache - lookup) could be amortized over only few values. Such small blocks were - immediately dismissed. + stencil would span more than one block. Finally, per-block overhead in + storage (e.g., shared exponent, bit offset) and computation (e.g., software + cache lookup) could be amortized over only few values. Such small blocks + were immediately dismissed. * When *n* = 8, blocks are too large, for several reasons: @@ -1364,7 +1378,7 @@ reasons why *n* = 4 was chosen: have to be decoded even if only one value were requested. * "Skinny" arrays would have to be padded to multiples of *n* = 8, which - could introduce an unaccepable storage overhead. For instance, a + could introduce an unacceptable storage overhead. For instance, a 30 |times| 20 |times| 3 array of 1,800 values would be padded to 32 |times| 24 |times| 8 = 6,144 values, an increase of about 3.4 times. In contrast, when *n* = 4, only 32 |times| 20 |times| 4 = 2,560 values From c0a3f9279d9116e2dca1031cba9ba6085ca2bc9e Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 27 Oct 2024 10:59:47 -0700 Subject: [PATCH 219/277] Add references to linear solvers that use zfp --- docs/source/faq.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 9c82a2f0e..d50d809da 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -260,6 +260,10 @@ A: Yes, but your mileage may vary. Dense matrices, unlike smooth scalar fields, rarely exhibit correlation between adjacent rows and columns. Thus, the quality or compression ratio may suffer. +For examples of dense linear solvers that use |zfp| for matrix storage, +see `STRUMPACK `__ +and `ButterflyPACK `__. + ------------------------------------------------------------------------------- .. _q-structured: From a46fa8b91bf2d69f4ffcf04af4f908383828ba79 Mon Sep 17 00:00:00 2001 From: lindstro Date: Sun, 27 Oct 2024 15:42:23 -0700 Subject: [PATCH 220/277] Add code example and FAQ on chunked (de)compression --- CHANGELOG.md | 4 + docs/source/examples.rst | 32 ++++++ docs/source/faq.rst | 83 ++++++++++++++- docs/source/installation.rst | 2 + examples/CMakeLists.txt | 3 + examples/Makefile | 4 + examples/chunk.c | 192 +++++++++++++++++++++++++++++++++++ 7 files changed, 318 insertions(+), 2 deletions(-) create mode 100644 examples/chunk.c diff --git a/CHANGELOG.md b/CHANGELOG.md index d8ca4459f..ca240fa72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,10 @@ Change Log ## Unreleased +### Added + +- A new code example, `chunk`, shows how to perform (de)compression in chunks. + ### Fixed - #241: Signed left shifts, integer overflow invoke undefined behavior. diff --git a/docs/source/examples.rst b/docs/source/examples.rst index c73084d0e..9391b8415 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -37,6 +37,38 @@ storage would not be enough to distinguish more than 16 different values. For more advanced compressed-array features, see the :ref:`tutorial `. +.. _ex-chunk: + +Chunked (De)compression +----------------------- + +The :program:`chunk` program is an example of how to perform chunked +(de)compression, where the compressed stream for a 3D array is produced or +consumed in multiple chunks. Chunking slices the array along the *z* +direction (the slowest varying dimension) into slabs that are (de)compressed +independently. Assuming the chosen array dimensions, rate, and number of +chunks admit (de)compression by satisfying certain constraints (see FAQ +:ref:`#32 `), (de)compression in chunks should result in the same +output as if the entire array were (de)compressed all at once. + +The array dimensions are specified as :code:`-3 nx ny nz` (default is +125 |times| 100 |times| 240); the rate as :code:`-r rate` (default is +16 bits/value); and the number of chunks as :code:`-n chunks` (default is one +chunk). Without :code:`-d`, a synthetic array is generated and compressed to +standard output. Using :code:`-d`, standard input is decompressed and written +to standard output. For example:: + + chunk -n 1 > single.zfp + chunk -n 4 > quadruple.zfp + diff single.zfp quadruple.zfp + + chunk -n 1 -d < single.zfp > single.f64 + chunk -n 4 -d < single.zfp > quadruple.f64 + diff single.f64 quadruple.f64 + +Here :program:`diff` should report no differences. See FAQ +:ref:`#32 ` for further discussion of chunked (de)compression. + .. _ex-diffusion: Diffusion Solver diff --git a/docs/source/faq.rst b/docs/source/faq.rst index d50d809da..9c800effa 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -43,6 +43,7 @@ Questions answered in this FAQ: #. :ref:`How can I print array values? ` #. :ref:`What is known about zfp compression errors? ` #. :ref:`Why are zfp blocks 4 * 4 * 4 values? ` + #. :ref:`Can zfp (de)compress a single array in chunks? ` ------------------------------------------------------------------------------- @@ -530,8 +531,9 @@ when calling the high-level API function :c:func:`zfp_decompress`. With regards to the :c:type:`zfp_field` struct passed to :c:func:`zfp_compress` and :c:func:`zfp_decompress`, field dimensions must -match between compression and decompression, however strides need not match -(see :ref:`Q16 `). Additionally, the scalar type, +generally match between compression and decompression, though see +:ref:`Q32 ` on chunked (de)compression. Strides, however, need +not match; see :ref:`Q16 `. Additionally, the scalar type, :c:type:`zfp_type`, must match. For example, float arrays currently have a compressed representation different from compressed double arrays due to differences in exponent width. It is not possible to compress a double array @@ -1418,3 +1420,80 @@ above factors. Additionally, *n* = 4 has these benefits: a compressed 3D block occupies 128 bytes, or 1-2 hardware cache lines on contemporary computers. Hence, a fair number of *compressed* blocks can also fit in hardware cache. + +------------------------------------------------------------------------------- + +.. _q-chunked: + +Q32: *Can zfp (de)compress a single array in chunks?* + +Yes, but there are restrictions. + +First, one can trivially partition any array into subarrays and (de)compress +those independently using separate matching :c:func:`zfp_compress` and +:c:func:`zfp_decompress` calls for each chunk. Via subarray dimensions, +strides, and pointers into the larger array, one can thus (de)compress the +full array in pieces; see also :ref:`Q16 `. This approach to +chunked (de)compression incurs no constraints on compression mode, compression +parameters, or array dimensions, though producer and consumer must agree on +chunk size. This type of chunking is employed by the |zfp| HDF5 filter +`H5Z-ZFP `__ for I/O. + +A more restricted form of chunked (de)compression is to produce (compress) or +consume (decompress) a single compressed stream for the whole array in chunks +in a manner compatible with producing/consuming the entire stream all at once. +Such chunked (de)compression divides the array into slabs along the slowest +varying dimension (e.g., along *z* for 3D arrays), (de)compresses one slab at +a time, and produces or consumes consecutive pieces of the sequential +compressed stream. This approach, too, is possible, though only when these +requirements are met: + +* The size of each chunk (except the last) must be a whole multiple of four + along the slowest varying dimension; other dimensions are not subject to this + constraint. For example, a 3D array with *nz* = 120 can be (de)compressed + in two or three equal-size chunks, but not four, since 120/2 = 60, and + 120/3 = 40 are both divisible by four, but 120/4 = 30 is not. Other viable + chunk sizes are 120/5 = 24, 120/6 = 20, 120/10 = 12, 120/15 = 8, and + 120/30 = 4. Note that other chunk sizes may be possible by relaxing the + constraint that they all be equal, as exploited by the + :ref:`chunk ` code example, e.g., *nz* = 120 can be partitioned + into three chunks of size 32 and one of size 24. + + The reason for this requirement is that |zfp| always pads each compressed + (sub)array to fill out whole blocks of size 4 in each dimension, and such + interior padding would not occur if the whole array were compressed as a + single chunk. + +* The length of the compressed substream for each chunk must be a multiple of + the :ref:`word size `. The reason for this is that each + :c:func:`zfp_compress` and :c:func:`zfp_decompress` call aligns the stream + on a word boundary upon completion. One may avoid this requirement by using + the low-level API, which does not automatically perform such alignment. + +.. note:: + + When using the :ref:`high-level API `, the requirement on stream + alignment essentially limits chunked (de)compression to + :ref:`fixed-rate mode `, as it is the only one that can + guarantee that the size of each compressed chunk is a multiple of the word + size. To support other compression modes, use the + :ref:`low-level API `. + +Chunked (de)compression requires the user to set the :c:type:`zfp_field` +dimensions to match the current chunk size and to set the +:ref:`field pointer ` to the beginning of each uncompressed +chunk before (de)compressing it. The user may also have to position the +compressed stream so that it points to the beginning of each compressed +chunk. See the :ref:`code example ` for how one may implement +chunked (de)compression. + +Note that the chunk size used for compression need not match the size used for +decompression; e.g., the array may be compressed in a single sweep but +decompressed in chunks, or vice versa. Any combination of chunk sizes that +respect the above constraints is valid. + +Chunked (de)compression makes it possible to perform, for example, windowed +streaming computations on smaller subsets of the decompressed array at a time, +i.e., without having to allocate enough space to hold the entire uncompressed +array. It also can be useful for overlapping or interleaving computation with +(de)compression in a producer/consumer model. diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 9009fa891..4598024d2 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -340,6 +340,8 @@ in the same manner that :ref:`build targets ` are specified, e.g., Default: undefined/off. +.. _word-size: + .. c:macro:: BIT_STREAM_WORD_TYPE Unsigned integer type used for buffering bits. Wider types tend to give diff --git a/examples/CMakeLists.txt b/examples/CMakeLists.txt index 73137223f..0bc9c5676 100644 --- a/examples/CMakeLists.txt +++ b/examples/CMakeLists.txt @@ -2,6 +2,9 @@ add_executable(array array.cpp) target_compile_definitions(array PRIVATE ${zfp_compressed_array_defs}) target_link_libraries(array zfp) +add_executable(chunk chunk.c) +target_link_libraries(chunk zfp) + add_executable(diffusion diffusion.cpp) target_compile_definitions(diffusion PRIVATE ${zfp_compressed_array_defs}) if(ZFP_WITH_OPENMP) diff --git a/examples/Makefile b/examples/Makefile index 0e288544c..6b4b1d100 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -2,6 +2,7 @@ include ../Config BINDIR = ../bin TARGETS = $(BINDIR)/array\ + $(BINDIR)/chunk\ $(BINDIR)/diffusion\ $(BINDIR)/inplace\ $(BINDIR)/iterator\ @@ -25,6 +26,9 @@ all: $(TARGETS) $(BINDIR)/array: array.cpp ../lib/$(LIBZFP) $(CXX) $(CXXFLAGS) $(INCS) array.cpp $(CXXLIBS) -o $@ +$(BINDIR)/chunk: chunk.c ../lib/$(LIBZFP) + $(CC) $(CFLAGS) $(INCS) chunk.c $(CLIBS) -o $@ + $(BINDIR)/diffusion: diffusion.cpp ../lib/$(LIBZFP) $(CXX) $(CXXFLAGS) $(INCS) diffusion.cpp $(CXXLIBS) -o $@ diff --git a/examples/chunk.c b/examples/chunk.c new file mode 100644 index 000000000..4da611a8c --- /dev/null +++ b/examples/chunk.c @@ -0,0 +1,192 @@ +/* code example showing how to (de)compress a 3D array in chunks */ + +#include +#include +#include +#include +#include +#include "zfp.h" + +/* open compressed stream for (de)compressing field at given rate */ +static zfp_stream* +stream(const zfp_field* field, double rate) +{ + const size_t bx = (field->nx + 3) / 4; /* # blocks along x */ + const size_t by = (field->ny + 3) / 4; /* # blocks along y */ + const size_t bz = (field->nz + 3) / 4; /* # blocks along z */ + + zfp_stream* zfp; /* compressed stream */ + size_t words; /* word size of compressed buffer */ + size_t bytes; /* byte size of compressed buffer */ + void* buffer; /* storage for compressed stream */ + bitstream* stream; /* bit stream to write to or read from */ + + /* allocate meta data for a compressed stream */ + zfp = zfp_stream_open(NULL); + + /* set fixed-rate mode with no alignment */ + zfp_stream_set_rate(zfp, rate, zfp_type_double, zfp_field_dimensionality(field), zfp_false); + + /* determine exact compressed size in words */ + words = (bx * by * bz * zfp->maxbits + stream_word_bits - 1) / stream_word_bits; + + /* allocate buffer for single chunk of compressed data */ + bytes = words * stream_word_bits / CHAR_BIT; + buffer = malloc(bytes); + + /* associate bit stream with allocated buffer */ + stream = stream_open(buffer, bytes); + zfp_stream_set_bit_stream(zfp, stream); + + return zfp; +} + +/* compress chunk */ +static zfp_bool +compress(zfp_stream* zfp, const zfp_field* field) +{ + void* buffer = stream_data(zfp_stream_bit_stream(zfp)); + + /* compress chunk and output compressed data */ + size_t size = zfp_compress(zfp, field); + if (!size) + return zfp_false; + fwrite(buffer, 1, size, stdout); + + return zfp_true; +} + +/* decompress chunk */ +static zfp_bool +decompress(zfp_stream* zfp, zfp_field* field) +{ + void* buffer = stream_data(zfp_stream_bit_stream(zfp)); + + /* decompress chunk and output uncompressed data */ + size_t size = fread(buffer, 1, stream_capacity(zfp_stream_bit_stream(zfp)), stdin); + if (zfp_decompress(zfp, field) != size) + return zfp_false; + fwrite(zfp_field_pointer(field), sizeof(double), zfp_field_size(field, NULL), stdout); + + return zfp_true; +} + +/* print command usage */ +static int +usage(void) +{ + fprintf(stderr, "chunk [options] output\n"); + fprintf(stderr, "Options:\n"); + fprintf(stderr, "-3 : array dimensions\n"); + fprintf(stderr, "-d : decompress (from stdin to stdout); else compress\n"); + fprintf(stderr, "-n : number of chunks along z dimension\n"); + fprintf(stderr, "-r : rate in bits/value\n"); + + return EXIT_FAILURE; +} + +int main(int argc, char* argv[]) +{ + /* command-line arguments */ + zfp_bool decode = zfp_false; + double rate = 16; + int nx = 125; + int ny = 100; + int nz = 240; + int chunks = 1; + + /* local variables */ + double* array; + double* ptr; + zfp_field* field; + zfp_stream* zfp; + int i, x, y, z, mz; + + /* process command line */ + for (i = 1; i < argc; i++) + if (!strcmp(argv[i], "-3")) { + if (++i == argc || sscanf(argv[i], "%d", &nx) != 1 || + ++i == argc || sscanf(argv[i], "%d", &ny) != 1 || + ++i == argc || sscanf(argv[i], "%d", &nz) != 1) + return usage(); + } + else if (!strcmp(argv[i], "-d")) + decode = zfp_true; + else if (!strcmp(argv[i], "-r")) { + if (++i == argc || sscanf(argv[i], "%lf", &rate) != 1) + return usage(); + } + else if (!strcmp(argv[i], "-n")) { + if (++i == argc || sscanf(argv[i], "%d", &chunks) != 1) + usage(); + } + else + return usage(); + + /* compute chunk size (must be a multiple of four) */ + mz = 4 * ((nz + 4 * chunks - 1) / (4 * chunks)); + if ((chunks - 1) * mz >= nz) { + fprintf(stderr, "cannot partition nz=%d into %d chunks\n", nz, chunks); + return EXIT_FAILURE; + } + + /* allocate whole nx * ny * nz array of doubles */ + array = malloc(nx * ny * nz * sizeof(double)); + + if (!decode) { + /* initialize array to be compressed */ + for (z = 0; z < nz; z++) + for (y = 0; y < ny; y++) + for (x = 0; x < nx; x++) + array[x + nx * (y + ny * z)] = 1. / (1 + x + nx * (y + ny * z)); + } + + /* initialize field, stream, and compressed buffer */ + field = zfp_field_3d(array, zfp_type_double, nx, ny, mz); + zfp = stream(field, rate); + + /* warn if compressed size is not a multiple of word size */ + if (chunks > 1 && (zfp_field_blocks(field) * zfp->maxbits) % stream_word_bits) + fprintf(stderr, "warning: compressed size (%ld) is not a multiple of word size (%ld)\n", (long)(zfp_field_blocks(field) * zfp->maxbits), (long)stream_word_bits); + + /* (de)compress array in chunks */ + ptr = array; + for (z = 0; z < nz; z += mz) { + /* compute current chunk size as min(mz, nz - z) */ + int cz = mz < nz - z ? mz : nz - z; + + /* set chunk size and pointer into uncompressed array */ + zfp_field_set_pointer(field, ptr); + zfp_field_set_size_3d(field, nx, ny, cz); + + /* reuse compressed buffer by rewinding compressed stream */ + zfp_stream_rewind(zfp); + + if (decode) { + /* decompress current chunk from stdin to stdout */ + if (!decompress(zfp, field)) { + fprintf(stderr, "decompression failed\n"); + return EXIT_FAILURE; + } + } + else { + /* compress current chunk to stdout */ + if (!compress(zfp, field)) { + fprintf(stderr, "compression failed\n"); + return EXIT_FAILURE; + } + } + + /* advance pointer to next chunk of uncompressed data */ + ptr += nx * ny * cz; + } + + /* clean up */ + free(stream_data(zfp_stream_bit_stream(zfp))); + stream_close(zfp_stream_bit_stream(zfp)); + zfp_stream_close(zfp); + zfp_field_free(field); + free(array); + + return EXIT_SUCCESS; +} From 64ac487e9b41fa3cc647402b172bddd062ab02b2 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 18 Nov 2024 13:41:52 +0000 Subject: [PATCH 221/277] Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 54c4313cd..a07e39502 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -42,7 +42,7 @@ jobs: lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml - name: Upload Report to Codecov - uses: codecov/codecov-action@v4 + uses: codecov/codecov-action@v5 with: files: ${{github.workspace}}/build/coverage.xml env_vars: Actions From 889d15b00070e75083cff2a5b107119c8d0fdc73 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 26 Nov 2024 08:52:52 -0800 Subject: [PATCH 222/277] Update comments to reflect support for 4D arrays --- include/zfp.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/include/zfp.h b/include/zfp.h index 5e87a3244..2f64126a3 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -631,11 +631,11 @@ zfp_stream_align( /* The functions below all compress either a complete contiguous d-dimensional block of 4^d scalars or a complete or partial block assembled from a strided -array. In the latter case, p points to the first scalar; (nx, ny, nz) specify -the size of the block, with 1 <= nx, ny, nz <= 4; and (sx, sy, sz) specify the -strides, i.e. the number of scalars to advance to get to the next scalar along -each dimension. The functions return the number of bits of compressed storage -needed for the compressed block. +array. In the latter case, p points to the first scalar; (nx, ny, nz, nw) +specify the size of the block, with 1 <= nx, ny, nz, nw <= 4; and +(sx, sy, sz, sw) specify the strides, i.e., the number of scalars to advance +to get to the next scalar along each dimension. The functions return the +number of bits of compressed storage needed for the compressed block. */ /* encode 1D contiguous block of 4 values */ From 54b36c875135e8bc6ad7b667ed1bda835d7cb7bc Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 13 Jan 2025 18:57:21 -0800 Subject: [PATCH 223/277] Add zfp_block_maximum_size() function --- CHANGELOG.md | 2 ++ docs/source/faq.rst | 5 +++++ docs/source/low-level-api.rst | 16 ++++++++++++++++ include/zfp.h | 3 +++ src/zfp.c | 28 ++++++++++++++++++++++++++++ 5 files changed, 54 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index ca240fa72..4e0bd5838 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ Change Log ### Added - A new code example, `chunk`, shows how to perform (de)compression in chunks. +- A new utility function `zfp_block_maximum_size()` returns maximum block size + for given scalar type, dimensionality, and compression mode. ### Fixed diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 9c800effa..a05b0fc25 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -1204,6 +1204,8 @@ it is useful to know the maximum number of bits that a compressed block can occupy. In addition to the :c:macro:`ZFP_MAX_BITS` macro, the following table lists the maximum block size (in bits) for each scalar type, whether :ref:`reversible mode ` is used, and block dimensionality. +Note that these sizes are upper bounds that are independent of compression +parameter settings, which may further constrain the storage size. +--------+---------+-------+-------+-------+-------+ | type | rev. | 1D | 2D | 3D | 4D | @@ -1225,6 +1227,9 @@ table lists the maximum block size (in bits) for each scalar type, whether | | |check| | 278 | 1058 | 4178 | 16658 | +--------+---------+-------+-------+-------+-------+ +The function :c:func:`zfp_block_maximum_size` returns the block sizes encoded +in this table. + ------------------------------------------------------------------------------- .. _q-printf: diff --git a/docs/source/low-level-api.rst b/docs/source/low-level-api.rst index 27a2ba7ce..796fa1489 100644 --- a/docs/source/low-level-api.rst +++ b/docs/source/low-level-api.rst @@ -365,6 +365,22 @@ appropriate bias for unsigned integer data. Convert *dims*-dimensional contiguous block from 32-bit integer type. Use *dims* = 0 to demote a single value. +---- + +.. c:function:: size_t zfp_block_maximum_size(zfp_type type, uint dims, zfp_bool reversible) + + Maximum compressed size in bits of a single *dims*-dimensional block of + the specified scalar type. Use *reversible* = :code:`zfp_true` with + :ref:`reversible mode `. Note that this bound does not + include any potential padding at the end of the stream to fill out a whole + number of words of length :c:var:`stream_word_bits`, which is inserted + when calling :c:func:`zfp_stream_flush` or :c:func:`stream_flush`. + Similarly, it includes no storage for the optional header. The storage + bounds returned by this function are known to be loose, and it is + possible that they will be tightened in future releases. This function + returns zero if any of the arguments are invalid. See also + :ref:`Q28 `. + .. _ll-cpp-wrappers: C++ Wrappers diff --git a/include/zfp.h b/include/zfp.h index 2f64126a3..a56326f13 100644 --- a/include/zfp.h +++ b/include/zfp.h @@ -788,6 +788,9 @@ void zfp_demote_int32_to_uint8(uint8* oblock, const int32* iblock, uint dims); void zfp_demote_int32_to_int16(int16* oblock, const int32* iblock, uint dims); void zfp_demote_int32_to_uint16(uint16* oblock, const int32* iblock, uint dims); +/* maximum number of bits/block of compressed storage */ +size_t zfp_block_maximum_size(zfp_type type, uint dims, zfp_bool reversible); + #ifdef __cplusplus } #endif diff --git a/src/zfp.c b/src/zfp.c index f13a4025a..ce3b759ba 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -102,6 +102,34 @@ zfp_type_size(zfp_type type) } } +size_t +zfp_block_maximum_size(zfp_type type, uint dims, zfp_bool reversible) +{ + static const size_t size_table[2][4][4] = { + /* non-reversible mode */ + { + { 131, 527, 2111, 8447 }, /* int32 */ + { 259, 1039, 4159, 16639 }, /* int64 */ + { 140, 536, 2120, 8456 }, /* float */ + { 271, 1051, 4171, 16651 }, /* double */ + }, + /* reversible mode */ + { + { 136, 532, 2116, 8452 }, /* int32 */ + { 265, 1045, 4165, 16645 }, /* int64 */ + { 146, 542, 2126, 8462 }, /* float */ + { 278, 1058, 4178, 16658 }, /* double */ + }, + }; + + /* check arguments */ + if (!(zfp_type_int32 <= type && type <= zfp_type_double) || + !(1 <= dims && dims <= 4)) + return 0; + + return size_table[reversible ? 1 : 0][type - 1][dims - 1]; +} + /* public functions: fields ------------------------------------------------ */ zfp_field* From 5289ca3f54185bd4754f8d0856b0bbbee3309fac Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 13 Jan 2025 19:03:20 -0800 Subject: [PATCH 224/277] Bump Ubuntu gcc version --- .github/workflows/tests.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 9a4642853..9013b25c2 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -13,8 +13,8 @@ jobs: matrix: include: - os: ubuntu-latest - cxx_compiler: g++-10 - c_compiler: gcc-10 + cxx_compiler: g++-14 + c_compiler: gcc-14 omp: ON target: all From 47cfabbf37c9c276c26c527f71090dc53369c99c Mon Sep 17 00:00:00 2001 From: Ryan Mast Date: Fri, 17 Jan 2025 13:45:43 -0800 Subject: [PATCH 225/277] Add native AArch64 Linux CI test jobs --- .github/workflows/tests.yml | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 9013b25c2..1248fd58c 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -24,6 +24,20 @@ jobs: omp: ON target: all + - os: ubuntu-24.04-arm + cxx_compiler: g++-14 + c_compiler: gcc-14 + omp: ON + target: all + architecture: arm64 + + - os: ubuntu-24.04-arm + cxx_compiler: clang++ + c_compiler: clang + omp: ON + target: all + architecture: arm64 + - os: macos-latest cxx_compiler: g++-12 c_compiler: gcc-12 @@ -43,7 +57,7 @@ jobs: uses: actions/setup-python@v4 with: python-version: '3.x' - architecture: x64 + architecture: ${{ matrix.architecture || 'x64' }} - name: Install zfpy dependencies run: | @@ -52,7 +66,7 @@ jobs: python -m pip install setuptools - name: Setup OpenMP (Linux) - if: ${{matrix.os == 'ubuntu-latest' && matrix.cxx_compiler == 'clang++'}} + if: ${{(matrix.os == 'ubuntu-latest' || matrix.os == 'ubuntu-24.04-arm') && matrix.cxx_compiler == 'clang++'}} run: sudo apt-get update; sudo apt-get install -y libomp5 libomp-dev - name: Setup OpenMP (MacOS) From bb76298b94d7a324e9b11162f387325e61b0255d Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 12 Feb 2025 10:15:42 -0800 Subject: [PATCH 226/277] Ensure inline bitstream tests include inline.h (resolves #252) --- tests/src/inline/testBitstream.c | 5 +++-- tests/src/inline/testBitstreamSmallWsize.c | 5 +++-- tests/src/inline/testBitstreamStrided.c | 5 +++-- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/tests/src/inline/testBitstream.c b/tests/src/inline/testBitstream.c index c025d95f9..29fe8151b 100644 --- a/tests/src/inline/testBitstream.c +++ b/tests/src/inline/testBitstream.c @@ -3,8 +3,9 @@ #include #include -#include "include/zfp/bitstream.h" -#include "include/zfp/bitstream.inl" +#include "zfp/internal/zfp/inline.h" +#include "zfp/bitstream.h" +#include "zfp/bitstream.inl" #define STREAM_WORD_CAPACITY 3 diff --git a/tests/src/inline/testBitstreamSmallWsize.c b/tests/src/inline/testBitstreamSmallWsize.c index 23350e553..290dd2924 100644 --- a/tests/src/inline/testBitstreamSmallWsize.c +++ b/tests/src/inline/testBitstreamSmallWsize.c @@ -5,8 +5,9 @@ #define BIT_STREAM_WORD_TYPE uint16 -#include "include/zfp/bitstream.h" -#include "include/zfp/bitstream.inl" +#include "zfp/internal/zfp/inline.h" +#include "zfp/bitstream.h" +#include "zfp/bitstream.inl" #define STREAM_WORD_CAPACITY 4 diff --git a/tests/src/inline/testBitstreamStrided.c b/tests/src/inline/testBitstreamStrided.c index d628d9eb0..3a946e139 100644 --- a/tests/src/inline/testBitstreamStrided.c +++ b/tests/src/inline/testBitstreamStrided.c @@ -5,8 +5,9 @@ #define BIT_STREAM_STRIDED -#include "include/zfp/bitstream.h" -#include "include/zfp/bitstream.inl" +#include "zfp/internal/zfp/inline.h" +#include "zfp/bitstream.h" +#include "zfp/bitstream.inl" // 4 words per block #define BLOCK_SIZE 4 From fbe66153e78b8d230922076f6ad8ce9dac9f54c4 Mon Sep 17 00:00:00 2001 From: Juniper Tyree <50025784+juntyr@users.noreply.github.com> Date: Sun, 15 Jun 2025 07:34:18 +0300 Subject: [PATCH 227/277] Fix zfp_stream_maximum_size overflow on 32bit target --- src/zfp.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/zfp.c b/src/zfp.c index ce3b759ba..583fbf3ea 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -766,7 +766,12 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) maxbits += values - 1 + values * MIN(zfp->maxprec, zfp_field_precision(field)); maxbits = MIN(maxbits, zfp->maxbits); maxbits = MAX(maxbits, zfp->minbits); - return ((ZFP_HEADER_MAX_BITS + blocks * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; + + uint64 maxsize = ((ZFP_HEADER_MAX_BITS + ((uint64)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; + + if (maxsize > SIZE_MAX) + return 0; + return (size_t)maxsize; } void From b599b8f37e6d0cb6350b1e99133c43bcd48be125 Mon Sep 17 00:00:00 2001 From: Juniper Tyree <50025784+juntyr@users.noreply.github.com> Date: Sun, 15 Jun 2025 20:10:14 +0300 Subject: [PATCH 228/277] Use bitstream_size instead of uint64 --- src/zfp.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/zfp.c b/src/zfp.c index 583fbf3ea..a82cc9432 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -767,7 +767,7 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) maxbits = MIN(maxbits, zfp->maxbits); maxbits = MAX(maxbits, zfp->minbits); - uint64 maxsize = ((ZFP_HEADER_MAX_BITS + ((uint64)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; + bitstream_size maxsize = ((ZFP_HEADER_MAX_BITS + ((bitstream_size)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; if (maxsize > SIZE_MAX) return 0; From 9036ae8c0df9f5a33c5651b9cf2f94be1a24f330 Mon Sep 17 00:00:00 2001 From: Juniper Tyree <50025784+juntyr@users.noreply.github.com> Date: Tue, 17 Jun 2025 10:29:25 +0300 Subject: [PATCH 229/277] Replace SIZE_MAX check --- src/zfp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zfp.c b/src/zfp.c index a82cc9432..93ec21b59 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -769,7 +769,8 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) bitstream_size maxsize = ((ZFP_HEADER_MAX_BITS + ((bitstream_size)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; - if (maxsize > SIZE_MAX) + /* check if the maxsize fits into size_t to avoid silent truncation */ + if (((size_t)maxsize) != maxsize) return 0; return (size_t)maxsize; } From 33891268d6cf2f04649e6f1a89158b755e2a28d1 Mon Sep 17 00:00:00 2001 From: Juniper Tyree <50025784+juntyr@users.noreply.github.com> Date: Wed, 18 Jun 2025 08:35:55 +0300 Subject: [PATCH 230/277] Move local variable declaration to the top --- src/zfp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/zfp.c b/src/zfp.c index 93ec21b59..ab1c8c949 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -744,6 +744,7 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) size_t blocks = zfp_field_blocks(field); uint values = 1u << (2 * dims); uint maxbits = 0; + bitstream_size maxsize; if (!dims) return 0; @@ -767,7 +768,7 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) maxbits = MIN(maxbits, zfp->maxbits); maxbits = MAX(maxbits, zfp->minbits); - bitstream_size maxsize = ((ZFP_HEADER_MAX_BITS + ((bitstream_size)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; + maxsize = ((ZFP_HEADER_MAX_BITS + ((bitstream_size)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; /* check if the maxsize fits into size_t to avoid silent truncation */ if (((size_t)maxsize) != maxsize) From 3090789f49fc5c75ae1f65f3e25f7c0ec2053727 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 24 Jun 2025 15:00:18 -0700 Subject: [PATCH 231/277] Discuss multiprecision paper in FAQ 13 on progressive streams --- docs/source/faq.rst | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index a05b0fc25..43fea0c87 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -478,6 +478,20 @@ implies that after every *m* 64-bit words have been decoded, the bit stream is advanced by *m* |times| *n* words to the next set of m 64-bit words associated with the block. +Rather than overcoming the technical challenges discussed here, we advocate a +simpler, compressor-agnostic approach to progressive-precision access: + +#. V. Magri, P. Lindstrom, + "`A General Framework for Progressive Data Compression and Retrieval `__," + IEEE Transactions on Visualization and Computer Graphics, 2024. + +Additional benefits of this framework include: + +- No need to perform surgery on |zfp|. +- Support for arbitrarily small |zfp| error tolerances (see + :ref:`Q17 `) and even lossless compression in the limit. +- Easy integratation with current file formats and I/O libraries. + ------------------------------------------------------------------------------- .. _q-init: @@ -1300,7 +1314,7 @@ resulting from |zfp|, as detailed in the following publications: SIAM Journal on Scientific Computing, 2020. #. P. Lindstrom, J. Hittinger, J. Diffenderfer, A. Fox, D. Osei-Kuffuor, J. Banks. "`ZFP: A Compressed Array Representation for Numerical Computations `__," - International Journal of High-Performance Computing Applications, 2024. + International Journal of High-Performance Computing Applications, 2025. #. A. Fox, P. Lindstrom. "`Statistical Analysis of ZFP: Understanding Bias `__," LLNL-JRNL-858256, Lawrence Livermore National Laboratory, 2024. From d32673117b2be3064c64e7840dc4dadb9fcd4710 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 24 Jun 2025 18:12:13 -0700 Subject: [PATCH 232/277] Fix zfp_stream_maximum_size() bug (resolves #270) --- CHANGELOG.md | 1 + docs/source/high-level-api.rst | 3 +++ src/zfp.c | 13 +++++++++---- 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e0bd5838..c12f216b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ Change Log ### Fixed - #241: Signed left shifts, integer overflow invoke undefined behavior. +- #270: Overflow of maximum stream size when `size_t` is 32 bits. --- diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 48ded1d4e..60282667b 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -568,6 +568,9 @@ Compressed Stream The buffer may then be resized (using :code:`realloc()`) after the actual number of bytes is known, as returned by :c:func:`zfp_compress`. + This function returns zero if the size exceeds what can be represented in + a :code:`size_t`. + .. _hl-func-stream: diff --git a/src/zfp.c b/src/zfp.c index ab1c8c949..475bff7cf 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -768,11 +768,16 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) maxbits = MIN(maxbits, zfp->maxbits); maxbits = MAX(maxbits, zfp->minbits); - maxsize = ((ZFP_HEADER_MAX_BITS + ((bitstream_size)blocks) * maxbits + stream_word_bits - 1) & ~(stream_word_bits - 1)) / CHAR_BIT; - - /* check if the maxsize fits into size_t to avoid silent truncation */ - if (((size_t)maxsize) != maxsize) + /* compute number of bytes in multiples of words */ + maxsize = ZFP_HEADER_MAX_BITS; + maxsize += (bitstream_size)blocks * maxbits; + maxsize = (maxsize + stream_word_bits - 1) & ~((bitstream_size)stream_word_bits - 1); + maxsize /= CHAR_BIT; + + /* ensure maxsize fits in size_t to avoid silent truncation */ + if ((size_t)maxsize != maxsize) return 0; + return (size_t)maxsize; } From 1e0640d87eeb80c8e48d37164f2cc864541c41a2 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 24 Jun 2025 18:25:32 -0700 Subject: [PATCH 233/277] Update reference to renamed factory.hpp --- include/zfp/array.hpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/zfp/array.hpp b/include/zfp/array.hpp index 07d5b08b6..4e2c3df52 100644 --- a/include/zfp/array.hpp +++ b/include/zfp/array.hpp @@ -14,7 +14,7 @@ class array { public: #include "zfp/internal/array/header.hpp" - // factory function (see zfpfactory.h) + // factory function (see factory.hpp) static zfp::array* construct(const zfp::array::header& header, const void* buffer = 0, size_t buffer_size_bytes = 0); // public virtual destructor (can delete array through base class pointer) From a59023905ade2494fb107f7c1809330b480d5e1c Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 1 Jul 2025 12:00:32 -0700 Subject: [PATCH 234/277] Add C templates for round_up, count_up --- src/template/template.h | 4 ++++ src/template/utils.c | 29 +++++++++++++++++++++++++++++ src/zfp.c | 23 ++++++++++++++--------- 3 files changed, 47 insertions(+), 9 deletions(-) create mode 100644 src/template/utils.c diff --git a/src/template/template.h b/src/template/template.h index fd5becf7e..e26ddbe36 100644 --- a/src/template/template.h +++ b/src/template/template.h @@ -2,6 +2,7 @@ #define TEMPLATE_H /* concatenation */ +#define _body(x) x ## _ #define _cat2(x, y) x ## _ ## y #define _cat3(x, y, z) x ## _ ## y ## _ ## z @@ -9,4 +10,7 @@ #define _t1(function, arg) _cat2(function, arg) #define _t2(function, type, dims) _cat3(function, type, dims) +/* 1-argument template instantiation; body must be defined in macro */ +#define _tdef1(function, type, args) _cat2(function, type)args _body(function)(type) + #endif diff --git a/src/template/utils.c b/src/template/utils.c new file mode 100644 index 000000000..e6e57332d --- /dev/null +++ b/src/template/utils.c @@ -0,0 +1,29 @@ +#ifndef ZFP_UTILS_H +#define ZFP_UTILS_H + +/* size / unit rounded up to the next integer */ +#define count_up_(type) /* (type size, type unit) */\ +{\ + return (size + unit - 1) / unit;\ +} + +/* smallest multiple of unit greater than or equal to size */ +#define round_up_(type) /* (type size, type unit) */\ +{\ + size += unit - 1;\ + size -= size % unit;\ + return size;\ +} + +/* template instantiations */ +static uint _tdef1(count_up, uint, (uint size, uint unit)) +static size_t _tdef1(count_up, size_t, (size_t size, size_t unit)) +static uint64 _tdef1(count_up, uint64, (uint64 size, uint64 unit)) +static bitstream_size _tdef1(count_up, bitstream_size, (bitstream_size size, bitstream_size unit)) + +static uint _tdef1(round_up, uint, (uint size, uint unit)) +static size_t _tdef1(round_up, size_t, (size_t size, size_t unit)) +static uint64 _tdef1(round_up, uint64, (uint64 size, uint64 unit)) +static bitstream_size _tdef1(round_up, bitstream_size, (bitstream_size size, bitstream_size unit)) + +#endif diff --git a/src/zfp.c b/src/zfp.c index 475bff7cf..f24ea7dbe 100644 --- a/src/zfp.c +++ b/src/zfp.c @@ -49,6 +49,10 @@ is_reversible(const zfp_stream* zfp) #include "share/parallel.c" #include "share/omp.c" +/* template instantiation of utility functions ------------------------------*/ + +#include "template/utils.c" + /* template instantiation of integer and float compressor -------------------*/ #define Scalar int32 @@ -769,9 +773,8 @@ zfp_stream_maximum_size(const zfp_stream* zfp, const zfp_field* field) maxbits = MAX(maxbits, zfp->minbits); /* compute number of bytes in multiples of words */ - maxsize = ZFP_HEADER_MAX_BITS; - maxsize += (bitstream_size)blocks * maxbits; - maxsize = (maxsize + stream_word_bits - 1) & ~((bitstream_size)stream_word_bits - 1); + maxsize = ZFP_HEADER_MAX_BITS + (bitstream_size)blocks * maxbits; + maxsize = _t1(round_up, bitstream_size)(maxsize, stream_word_bits); maxsize /= CHAR_BIT; /* ensure maxsize fits in size_t to avoid silent truncation */ @@ -799,8 +802,9 @@ zfp_stream_set_reversible(zfp_stream* zfp) double zfp_stream_set_rate(zfp_stream* zfp, double rate, zfp_type type, uint dims, zfp_bool align) { - uint n = 1u << (2 * dims); + const uint n = 1u << (2 * dims); uint bits = (uint)floor(n * rate + 0.5); + switch (type) { case zfp_type_float: bits = MAX(bits, 1 + 8u); @@ -811,15 +815,16 @@ zfp_stream_set_rate(zfp_stream* zfp, double rate, zfp_type type, uint dims, zfp_ default: break; } - if (align) { - /* for write random access, round up to next multiple of stream word size */ - bits += (uint)stream_word_bits - 1; - bits &= ~(stream_word_bits - 1); - } + + /* for write random access, round up to next multiple of stream word size */ + if (align) + bits = _t1(round_up, uint)(bits, (uint)stream_word_bits); + zfp->minbits = bits; zfp->maxbits = bits; zfp->maxprec = ZFP_MAX_PREC; zfp->minexp = ZFP_MIN_EXP; + return (double)bits / n; } From 2bb955b66bfe6737a6ee777c89d6f889ef596fd8 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 2 Jul 2025 11:09:35 -0700 Subject: [PATCH 235/277] Silence warnings due to unused templates --- src/template/utils.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/template/utils.c b/src/template/utils.c index e6e57332d..67e468e65 100644 --- a/src/template/utils.c +++ b/src/template/utils.c @@ -16,14 +16,14 @@ } /* template instantiations */ -static uint _tdef1(count_up, uint, (uint size, uint unit)) -static size_t _tdef1(count_up, size_t, (size_t size, size_t unit)) -static uint64 _tdef1(count_up, uint64, (uint64 size, uint64 unit)) -static bitstream_size _tdef1(count_up, bitstream_size, (bitstream_size size, bitstream_size unit)) +/* static uint _tdef1(count_up, uint, (uint size, uint unit)) */ +/* static size_t _tdef1(count_up, size_t, (size_t size, size_t unit)) */ +/* static uint64 _tdef1(count_up, uint64, (uint64 size, uint64 unit)) */ +/* static bitstream_size _tdef1(count_up, bitstream_size, (bitstream_size size, bitstream_size unit)) */ static uint _tdef1(round_up, uint, (uint size, uint unit)) -static size_t _tdef1(round_up, size_t, (size_t size, size_t unit)) -static uint64 _tdef1(round_up, uint64, (uint64 size, uint64 unit)) +/* static size_t _tdef1(round_up, size_t, (size_t size, size_t unit)) */ +/* static uint64 _tdef1(round_up, uint64, (uint64 size, uint64 unit)) */ static bitstream_size _tdef1(round_up, bitstream_size, (bitstream_size size, bitstream_size unit)) #endif From 3c58cb4082a34cbafc9a12c7e1f2b71c5272c16a Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 2 Jul 2025 11:10:17 -0700 Subject: [PATCH 236/277] Output word size in testzfp --- tests/testzfp.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/testzfp.cpp b/tests/testzfp.cpp index 9a547c487..6e0db89e1 100644 --- a/tests/testzfp.cpp +++ b/tests/testzfp.cpp @@ -1115,6 +1115,7 @@ int main(int argc, char* argv[]) break; } std::cout << std::endl; + std::cout << "word size " << stream_word_bits << std::endl; std::cout << std::endl; uint sizes = 0; From 0e9034eb9098bdaf06309f2b6d3f1f688007b291 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 7 Jul 2025 17:35:49 -0700 Subject: [PATCH 237/277] Add detailed section on stream headers --- docs/source/execution.rst | 2 +- docs/source/high-level-api.rst | 136 ++++++++++++++++++++++++--------- docs/source/limitations.rst | 2 +- docs/source/tutorial.rst | 2 +- 4 files changed, 104 insertions(+), 38 deletions(-) diff --git a/docs/source/execution.rst b/docs/source/execution.rst index 7a69cf2d6..02c0137d5 100644 --- a/docs/source/execution.rst +++ b/docs/source/execution.rst @@ -200,7 +200,7 @@ The CUDA implementation has a number of limitations: * Only the :ref:`fixed-rate mode ` mode is supported. Other modes will be supported in a future release. * 4D arrays are not supported. -* :ref:`Headers
` are not supported. Any header already present in +* :ref:`Headers ` are not supported. Any header already present in the stream will be silently overwritten on compression. * |zfp| must be built with a :c:macro:`ZFP_BIT_STREAM_WORD_SIZE` of 64 bits. * Although :ref:`strides ` are supported, fields must be contiguous diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 60282667b..1d13c0df6 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -26,6 +26,7 @@ The following sections are available: * :ref:`hl-func-exec` * :ref:`hl-func-config` * :ref:`hl-func-field` + * :ref:`hl-func-headers` * :ref:`hl-func-codec` .. _hl-macros: @@ -227,16 +228,6 @@ bitwise ORed together. Use :c:macro:`ZFP_DATA_ALL` to count all storage used. All storage (bitwise OR of all :code:`ZFP_DATA` constants). ----- - -.. c:macro:: ZFP_ROUND_FIRST -.. c:macro:: ZFP_ROUND_NEVER -.. c:macro:: ZFP_ROUND_LAST - - Available rounding modes for :c:macro:`ZFP_ROUNDING_MODE`, which - specifies at build time how |zfp| performs rounding in lossy compression - mode. - .. _hl-types: @@ -1010,6 +1001,105 @@ Array Metadata Return :code:`zfp_true` upon success. See :c:func:`zfp_field_metadata` for how to encode *meta*. +.. index:: + single: Header + +.. _headers: +.. _hl-func-headers: + +Stream Headers +^^^^^^^^^^^^^^ + +When decompressing data, |zfp| needs sufficient information to determine +:ref:`compression mode ` and settings (like rate, precision, or +accuracy) as well as the underlying type and dimensions of the uncompressed +array. The compressed stream itself does not store this information unless +the user encodes it in a short, optional header. The functions below allow +writing such headers on compression and later reading them during +decompression. + +|zfp| headers have been designed to be very concise representations of +:ref:`compression parameters ` and +:ref:`array metadata ` to accommodate compression of many +small (sub)arrays or spatially varying compression settings with low storage +overhead. In most cases, 64 bits of header data suffice to describe both +array and compression parameters. + +Headers are divided into three separate fields, each of which is optional +and included in the header by passing a :ref:`bit mask ` to +one of the functions below: + +* :c:macro:`ZFP_HEADER_MAGIC`: A 32-bit "magic" constant for identifying the + stream as |zfp| data. + +* :c:macro:`ZFP_HEADER_META`: A 52-bit metadata field that specifies the type + and shape of the uncompressed array. This field encodes one of four scalar + types (32- and 64-bit integer and floating-point types), one of four + dimensionalities 1 |leq| *d* |leq| 4, and the array dimensions (or shape). + The array dimensions supported are limited to 48 / *d* bits each. For + example, when *d* = 2, 24 bits are used to encode the number of rows and + columns, which each must be at most 2\ :sup:`24`. If dimensions exceed + these limits, an error code is generated. + +* :c:macro:`ZFP_HEADER_MODE`: A 12- or 64-bit compression settings field. + This variable-rate encoding has been designed to support most common + compression parameters through only 12 bits, which when combined with the + 52-bit metadata supports magic-less headers of 64 bits (perhaps for many + small subarrays with their own sizes and compression settings). The + following compression modes and parameters are supported in the short 12-bit + encoding: + + - :ref:`Fixed-rate mode `: 1 to 2048 bits per block of + |4powd| values. For example, all possible rates up to + 2048 / 4\ :sup:`3` = 32 bits/value are supported for 3D arrays. + + - :ref:`Fixed-precision mode `: 1 to 128 bits of + precision, which covers all precisions currently supported by |zfp|. + + - :ref:`Fixed-accuracy mode `: All tolerances in + the range [2\ :sup:`-1074`, 2\ :sup:`843`] |approx| + [5 |times| 10\ :sup:`-324`, 6 |times| 10\ :sup:`253`]. + Note that |zfp| error tolerances are limited to integer powers of two + (|zfp| will round down other tolerances). + + - :ref:`Reversible mode `: Takes no compression parameters. + + If compression parameters do not respect these constraints, then a longer + 64-bit encoding is used that supports all |zfp| compression modes and + parameter settings, including any :ref:`expert-mode ` settings + not covered by this list. A reserved 12-bit code is used to denote that + 52 additional bits follow for a full 64-bit encoding. + +.. note:: + Stream headers are not necessarily comprised of a whole number of + :ref:`words `. Hence, the payload compressed data may not be + word aligned when headers are prepended. + +The |zfp| high-level API provides the following two functions for reading and +writing headers, which should be called before (de)compressing data so that +the :c:struct:`zfp_stream` and :c:struct:`zfp_field` objects can be properly +initialized before decompression. + +.. c:function:: size_t zfp_write_header(zfp_stream* stream, const zfp_field* field, uint mask) + + Write an optional variable-length header to the stream that encodes + compression parameters, array metadata, etc. The header information written + is determined by the bit *mask* (see :c:macro:`macros `). + Unlike in :c:func:`zfp_compress`, no word alignment is enforced. See the + :ref:`limitations ` section for limits on the maximum array + size supported by the header. The return value is the number of bits + written, or zero upon failure. + +---- + +.. c:function:: size_t zfp_read_header(zfp_stream* stream, zfp_field* field, uint mask) + + Read header previously written using :c:func:`zfp_write_header`. The + *stream* and *field* data structures are populated with the information + stored in the header, as specified by the bit *mask* (see + :c:macro:`macros `). The caller must ensure that *mask* + agrees between header read and write calls. The return value is the number + of bits read, or zero upon failure. .. _hl-func-codec: @@ -1039,30 +1129,6 @@ Compression and Decompression those used during compression (see :c:type:`zfp_field`). This function further requires that :c:type:`zfp_stream` is initialized with the same :ref:`compression parameters ` used during compression. To assist - with this, an optional :ref:`header ` may be prepended that + with this, an optional :ref:`header ` may be prepended that encodes such metadata, which must be explicitly read using :c:func:`zfp_read_header` to initialize *stream* and *field*. - ----- - -.. _zfp-header: -.. c:function:: size_t zfp_write_header(zfp_stream* stream, const zfp_field* field, uint mask) - - Write an optional variable-length header to the stream that encodes - compression parameters, array metadata, etc. The header information written - is determined by the bit *mask* (see :c:macro:`macros `). - Unlike in :c:func:`zfp_compress`, no word alignment is enforced. See the - :ref:`limitations ` section for limits on the maximum array - size supported by the header. The return value is the number of bits - written, or zero upon failure. - ----- - -.. c:function:: size_t zfp_read_header(zfp_stream* stream, zfp_field* field, uint mask) - - Read header previously written using :c:func:`zfp_write_header`. The - *stream* and *field* data structures are populated with the information - stored in the header, as specified by the bit *mask* (see - :c:macro:`macros `). The caller must ensure that *mask* - agrees between header read and write calls. The return value is the number - of bits read, or zero upon failure. diff --git a/docs/source/limitations.rst b/docs/source/limitations.rst index fc0dd37cd..c3ee71dc2 100644 --- a/docs/source/limitations.rst +++ b/docs/source/limitations.rst @@ -20,7 +20,7 @@ that will address some of these limitations. extensions to other floating-point formats should be possible with minor effort. -- The optional |zfp| :ref:`header ` supports arrays with at +- The optional |zfp| :ref:`header ` supports arrays with at most 2\ :sup:`48` elements. The |zfp| header limits each dimension to 2\ :sup:`48/d` elements in a *d*-dimensional array, i.e., 2\ :sup:`48`, 2\ :sup:`24`, 2\ :sup:`16`, and 2\ :sup:`12` for 1D through diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst index e43e249ed..4846774b7 100644 --- a/docs/source/tutorial.rst +++ b/docs/source/tutorial.rst @@ -148,7 +148,7 @@ the compressed data, or via the :c:func:`zfp_write_header` and :c:func:`zfp_compress` and :c:func:`zfp_decompress` calls, respectively. These calls allow the user to specify what information to store in the header, including a 'magic' format identifier, the field type and dimensions, and the -compression parameters (see the :ref:`ZFP_HEADER ` macros). +compression parameters (see the :ref:`headers` section). In addition to this initialization, the bit stream has to be rewound to the beginning (before reading the header and decompressing the data):: From a72773860a03da2f58245f1f4b593ea8321735ce Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 7 Jul 2025 17:36:36 -0700 Subject: [PATCH 238/277] Fix bounding box of views figure --- docs/source/view-indexing.pdf | Bin 121307 -> 102197 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/docs/source/view-indexing.pdf b/docs/source/view-indexing.pdf index d6e2edb7f3df939b03385ffeecd7d769c10948b3..af43769737e7b399c10fa60dfe1dc42e082eec8b 100644 GIT binary patch literal 102197 zcmeFa2Ut`4w)P7Gf;6e3)JRbfA(s{=ksJeTXxEvc}ITvGkC{LGG9J@WmO?S7(vO`(ox?{ ziGaW$?p98eGBT7!DM1hAg^BDec7{{hy4n2P0lb71 z6TuQSI^$srf$Ce@+fqW6+}(U^-F&=3zOPr3M_Y(4RwDlMEu(1>PxiPYO zU0+u#kknuMYq;6DD?3{IK*Y8ySHx5;DGHH5z`?39PY+37-^<)CKqR*}Q+0(y0OT?R7pkhO2n0Z`Hq>PV z1o*jLRb53P0CKgXZWn_9KNq&Dt2hMMUajE15)k0$Dp7TngaALw0Ht6MV7m%%Bk;SgBozOJSYN3>w_Y2bOE3U99;n@0!LQ>ionqofFdI3 z3P2GNbOoS@2)Y7LLw0B49uI3o-x^L015ZAkY&T}j;;U{kw8}fihzF80#F3`H{s_afGPkCT>~fsz|cj2 zI>=V{*Op*Qf}zU*l>ivJ4p0h!p$h@E02sOwq!<80*Md|7VCZU)asZ|^BJ}`FcSH&T zmY1P>fYmiYO#rKJf|>wU z=L9tYtlkM~0$AM>)C92lC#VTvbx=?f!1Pe0CV=UpNKF9KN0FKUrjsHy0ZcDNY66&U ziqr%!{S>JQU^*&N6TtLTq$Ysrsz^-$(^rw20H(7dH33X-MQQ?=?uyg|F#Q#&31B)b zQWL=RSfnO^>9R;o0MloYngFKLA~gX_uSIGCm~M;I1Tg&;sR>{@E>aV~^jxGSfa$tO zO#st(k(vOe^CC3?Oz%Z%0+{ZL)C4g77pVzgIxtcb!1Q3GCV=U}NKF9Khmo29rV}GI z0ZcDOY66&UjMM}${TQhUU^+5V6TtLjq$Ysr%1BKB)0dH&0H!k|H33X-Mrs0>?u^s~ zF#Q>+31B)jQWL=RXrv~9>C#9|0Mn{@Hc}J7 z^lYRifa%&uO#sukk(vOeb0akYOz%c&0+{ZN)C4g78>tCkIyh1j!1QpWCV=VUNKF9K z$B~)iXqnf#-HW-Q}A{PKL`{ zMFx=a%6~W8-#xR+>zZ)qX8NAswgR73wuVuS$CLHS&C67fPe#%``Oo=^rQb+BJ9v5Z<)`l#7G*z(1zMCH{G__u zWmvalbE_sf&i%~WuJR0bHrYqL&>rCdSwYzWa2vzs(yLQFCFIKwp13~?9_YPQ^l()E z(|69r-d3sSYJ7f1Ur5pdAG-R)j>I{ido6m#KO4M!F=O1byf3TG+chVq?Omq7=V;U( zq3<_tb&IRBJ=izBmhYY;+Wv6oGG9q6gT_O_PF1a;kfeL_XK&owx;?hwm6$j6;D@F{ ziFtL2d&rOM$+c>xO85#q@bG}bMW(rcum-bhOQTagy~_q{eZ|@vluueoq)TWc3xZbqZgAdU zjLoQx)roAFd^Id!zK=$elPxhipC?vF?bXKN*`K#`RIx1!J)N#tJ7m_A6D>ZK6)o;p z_*MF>c>`NH%iPUM=+IRi*?~yS{%)DMM!glzr$N;$)6QqCm715cu8SWJtQvP}a+^?z zbP|m7QIN8eBf{r zS@OaA_niWZt`{8BIVkRV?&;1nRzsOb>YGt3wp(eB9!;1tzPcT% z)x~$}@LJ~F)R79M?0qrU94^kQBQgt5$Y(o7$2qVv@~4e(GwFEmipnpeR%t(VAzMIL z0=$hPDqnIpcwIglFb7n#9ks7HuwKzeH1dJ>!NKjxqVgsD!0YtjWlnHkB9(S_p4RMg zt5D_k6Uy0+hZR~8DOnC_g8GR1%VF7$F}EDj=E3dB!2S1vkFAi+x7f`H7NDh$xM~aD zNXew**$VEY25xi)yuJf`V6dQc`Bnr=oWtf`4fSl`iJwv$lI-q*y_O3ea3d28OZ&!m^_o?TKF zIR)M<%+tF5ba6OT+332-68JGlI8*I-^Pk9qYL%Ft;^896;$ovimY~S>TMQt@%CRToO)Gt`O}em z_Re+wkG|e+&QXlbyUebYkejS}B~R2I_PjRNHO!gPGF?mbxLkn?d0a+va&zI7Y9Y9j zvwbaJ_c_FfiEv}Ppz)#3=~Kod-;zEwJf4>&-W@p}?7F5!-BXbKyl-sbT;_@d5tZ}N zSYz<8*)6S(_Q@kV@>VGpdI}_HJH1*T9>33E-o>=XNi@p;?3Ryde{uWTMn_S8?*U%9 z&cspO`$Bg9$~tx|8C=1O@^?0?RwfMD$=``aJn=i0@5M0JcHGlYc9i~V#2|N+!wXWL zyWh0E_NgKUjSRaZ^3)I?0`eb|7946;NFFpAF-c|#@=FYi8JT;ZVbI(f8tV1LLRr!j z_JVzpEHgMSE{B-v{n4=EVuTlB@V@pbx89zUUn+W;ip%S4%LME~G?Lx#Irv@Qdp(YR z*x^MuxU;IH>7yFt&aMZ{p1UK~v?~?{9K_OW9BW!95O3xxxf;z9?r(hPagEO_P%9ol zL?zB0a}Y~cje}cUr2n9Fq%pbu((M;L-mLT=7zG;h+b=oD@8;Pvd6s-{>v?Les4AJ_ zd4y&ZG0$Ovo!XCb_rhw~8_A$BHJYZK9j{x@lX@d=MM^sDvfu4AUhr{7U$Cni?{QRN;%HnIK>2Qlt`Nt zF#G6jTv25cDZ}X~mu+#^>rcOHImyA8j2=o!zr)-)K(BqBkxP2x|`b z;r6mv;9zZ0f`{*yn4FXI(Rqj8i){G`3rjw!E|R5wOlQA*IJ$T-Wv2d!vzp#c)`F?Z z$&h2#ip$MoJwd)-mvYXM2nCo(IP|}%s2H4g%z7HP*s^Lr=~DL0;YEWzo9phI>-#>k zI&Ksf(oc9auPTe1nLQfm1?_99YzOO{TJ}%CVH9e1GBwF zH`HWHVRv5rv^SNZETf1P@{O1RI^ke(B||bTSuGJBk%(&uGDcY~DkZY(@z7^(4KLoNmaHVihNU$G4^(%`{{ zJW@%(hQVh2E4CR%T0Gd0hja!RAfOxEl{PA>Bg197dkodP#!YvtXDPEQRrnN2dB4Tw{-pLc&Bub|^Q?L+8c``WQB165Y z`hWYMgWrWr|5s$8432n^{Rz{-??R^jE3yOzCp^ghgvsD{Av^IavRnpdJjnipso-}Z zQ~nj%QwA42$o_?ta zJjnipDd2Y@Q~VX#3WEn8WPid0@Vk&1{DMrHlED)XGJ^X@yZ_fX?mrBll|_Ack>LJO z1Ji39_n(N*%A!8tNkF!Z49ETF;j^-+55W?UZ6m{R|0VdWEb4>81Z3ODaNK_bJ}Zm* zFf{?$HZmOd-;K}8qCRj=K(>tx$Ni7uv$CiUWd2mWZTGob^p>- z_^d4I%N+z{+sJU-KRbgLuKP!Q0fvAKg{&LL{U_nGvZyZ!5s+;o!*T!l_^d4Ii&O+; z+sJU-eR-iHGEbU_2pLrvTbC)x_@bEd{!3q1z`fRZDcs^p97zjMSY2yfNUEXj{6s8 z@Wyrjs4tomkfD&(;kf?-d{!3qWqbm%ZDcs^{~10ji#lNe0ogV(9QXeQpOrtx$Ni7vv$Cj@gAkByBg1k3>-elJ>I5nTWZTGob^p>d_^d4I zBr^nL+sJU-KPNsbi#ibx0ogV(9QO~!XJt_*DAf5K;FQ72C$AlpWUR-i4SZG>brL=TvTbC)x_@a}d{!29qCx_)ZDcs^p9`OrMV-u%fNUEXj{AqZG6qWZTGa+`j>XFRuGXoj8?%42A4Hj{ARy&&r}sE=xeRjSR>A z_u{j%s1xuKkZmKwasQL}tSssz#RO#A$Z*{MCO#{RI*~I0**3CY-M{p1d{!29vTFje zZDcs^AA--yqD}}-K(>tx$NdBNtSsuJ=>%lk$Z*`hB0ejNIL%A!ub zPe8Vf49ET3F!tx$Nd}Ov$CiFCjzoR+1ZhTf2 z6--D#wv7zO{fpqUvZ(Mz0m*)}p9_iu;K z%A$f&3COmQ;kbW)d{!0}CQCrJjSR>AUt$Qrb^oZqT>>%`vbJB{zcevEE4v*vPhhsq z49EU6;PbKoD!Q4#Y?~R5|L4JHX2C4SKOkr5H=V$^4QBvA;B&J8Dg>Oc&bHSX&IF)> z&&~p@flhGMmX(1`t0IuI72`HK1U0{2R0POuV@_4dZZ z+tt6C%HUC%1o^*8L~se*I2v|1ho_jI<|20n2J(qJaNlUN`V3i~_%Nz5T<$Zs1QC z2Yx5Ff0)>fvg7}^kL$ma+dqu#2L9z_0KkML!P9Vv3yX>X|IJ}a6db0ItNz~)Q@@kj zKg{g@BQk-1C%1nX+717ggTf!X-u_`~H~deS7k<-Q1^+O%8-V`_v%>FUhCWjCSJXM) z+-?B=C(H@Iiy3;B|BD&k;BEl^C(H=Hi`lPl<4NI7?grq0!hG<%nEm=To)q5bZUFu# z%m%-U*{^ToN#V`z2H^klt%Cnie&csBLm&M9tDoTw?*`z1!c6eHnEm=To)q5nZh(*y zj6lDQ*YfMzcv5)dy8%K>FcvfP8!bP7H}U6p1B9Gl46}b2;0+Lhg0Yz4%mH{4ya7T| zFcveMIRI~jH$aFA#$tvu2jI={1_)WfSj=$d0K6gI03j?Giy6)wfH%b(AfyFjF~gYy z@Wyxpgt%ZVW;k;I-W+d$kQa=_?AJK}e~>po2n@zzhBF7?P4Wf^iNP3V|1ioMAVdaZ zF~gYy@Md`fgv?+pW;k;I-Y{=~5E_id3}+6&o8}D=QiHLW;miSegOs9Dq018zAHdV==>-1MmiW z1B3u!EM_=!0N!M8fRG@J#q8HP0DrVMK!^~=VumvZ;LY|12pPgy%y8xayy4yeAw(F9 z8O|JlH{BZ`qzGe}{lj=~fDj{$#SCW-z?<(45ORdEnBmL;cmuuxLXa>PGn_d9Z^Ab~ zND{_khBF7?jrax#QNmcvew_pGXM6*MEMY8WICB8rkZ*txCXB@lXAZ!d@(mEugt3_6 z%mH{~z5zmTF~gYy@CJSZgkWJTW;k;I-o$T!kSvU0_75Y!0YbDe7BieV0B`0uK*$!x zVumvZ;0^r-2;st5%zm8%@TYzQgmhsnW;k;I-q>$|5HF0y3}+6&oBItA@`bUO;miSe zgTDbnz%UjwoH+n*@;5+87{+3TGY8;}{sssU!&uC4<^a6e-vA+F7{lxzhJOQukYOxl zzs>>p)4u^i$}kqQU*`b)@!tR;W*CbZ&K!U@{~I9W3}Z3FnFH_!fCGe}VJv1ia{%51 zaDb3BjKvIR4!|1$4iKV-v6$h^0eCaO;V{@AkT3I_&b9gyXAZy{0uCoc4P&h{oH+n* z3OJk)HH=~Q4`aaLgs5RGW;k;I-WYH=A!-z#9V&2mWQ2*q;|Vj5cTgk1^nI;9rJ_{dvI9o$RlaX1p=r@PC^&{J%5?91i@; z^sql~fuXJcUkeOx3^@E>#|y*7gpub-003bzxcGlBfD??egrj0W|9t=_7=7@c!{(J_ zt-Y-yhE53=&UjK=#PgB$pbxa~BYj#_C{<%7&Y+(mJzJHv0HVaFvV z6yBj?(rmi;`rw5jK?N6Yo3d^?s4xlHwJW`4r<9%%ofzR0#fKUnRKv&8#nD>9&ECZp0xknX zA6wVs5OMIyocOs90)c!8Z~++F`at-hs0V`DIr`Z`?c9C6pl*(CworHQpSP`_tsB(a zF#zi0;ALwI_3?Ly2HJYL3v8FF>Ie$JQ9*=%XYrSNfm=!9nUQ~fJl6~rV02#fr` z{fPw)*ERgt&~Sba@L>bry#rSt_~#J@o*z?ORN~)a6GHx`{l^7p3AKV+L#^Fi+})ry zP+O=Y)EVjmb%VM?J)mAtAE-a{%-@$BGL!J{OYW}((UTB3_zhmTol|w9EL!?0umy~pQ4~Qfc)G}LJYhGE+!@n zhaf~Hg+(D^;t24vC|m;ER|0$l@D|iI;&3>4yBPRW9P;zB2n@L;cm{YeF>$0O_$WwK zafl>n7q%-9hlol_fVIKF&(0$xB*99sF8zEa2r&t8KT#MQtOOwfTI0W6hD%C0bDqua51o(qppD~2t57eU)R7@|JOx`1b8Y0u$MTx z_}F?upsFsGKDNrX*6ud8fBWlv)t}5L(YZ^^pbgST-3Rjt%k|OfA|jy%NJNxU!o$NM z>ZQ2`p^(rvh&sOjpFrQ$#DM^r*^K3_uGOl=`=ZUog?(#POO>6EM8?@>1*Kk2yl{f? z$D6AY@zdyc0&s>MJI>3~6GQa(#@_7P8|p*874fy`#`(__6x4?{xc0pDfKgn2X(X?n zm_m}g^)IaI5Y z?D9h59;SuGMP{Ni5K;>d`!jBG=b@ydJ7A#M=o&%dRGwYBX0ekL@l`s&jVwi)70c3OFL`R~&n zA%RzGa1|C)nXX(cR3p3eW8 z?AyWEHU%P46%rzf51%tP_OlPt>G3KO5qoZZ$Qdkz-3IO{awJ)Da$4{+Tv)y*$4tRK zaXziQdN4C`d}qXpBFR>KIz(Tsz}!_L@GPZVxMD@c3)Vv+A8$b_LSXhV2FOm*9efwK zi5{^KO?(^V{vmjwsQTcK-FYq0y&sf3ZXJBjI2i6k!cOhEBgZV@0>!o6d(VfT4GSaL zGMsharKe6YNZ~2JW8y^<9e@9k)}yfQ(b8zC5Q)0o7Q2*N^(cte7uPpzuUcK_p&pys zie2GSJA6AU>%qOOx;6WEA%aPX7U#~(pYz;LRQ+f#9fXgM>>6Fd4$>{@XgP{G`+ZxV z?-Kil66c;XzQWpW&b+p+LHzK@av0mp^oi#%CeIf{4?_an$e2i^Zk-F?Y>wMjrMqVCPj%j$f?^%Y z;Z(`qtG`UXL*?xeBa4oub?LlA9mFF)oEyXI>xdx|>_f5S%AOi0w5jR%!#}F2JsRLb z?4+QSw^UE0BKlwktJ$@)Va+$6|KN)Op{&7yOBC`gyF#tz2F}AM&!4H?BIYFu)i?4D zA;~r%Qs@qOeY-c5;?eH0xgTuwD*O30w}hyON#le)W{?530<*#QYxMIx}w}o>BsjNd?MN{Ke}(1 zH!BBR0VwoklWj^*b=|e{rZ#?cv+MoYnR%K(cHY@jqrozbySJF%)xFmns9GFe zSjxS^&g|&>;gYdRTOp>C;Pxe28L>X#3i)~E=9^P(%NZ(3QJ3st9kK^+6&Pd< zI3)Cby34?u8@APl+n@M!`$38B} z3R$ZU{*<(~kN8xxkg>5!TfSR0b~wz4-S53XX-P0e zWJ~7#nfa1cVZCOTfG)!Y2~#*Z^x{5wU)qq|+zH+@T9qjZg0Aw;mjhPBUL7l5Z@lTsoZUqi0E?V)#HEdRfGCPW<7DE<%Lzkfi?^r+ShM zzfd+kna4}q!=5$Ks0|eB0~ft-+!BMR8$9{)}4k5x*)gLjt)b(iWqD(~iGlu}_r~R3?S5r#Uoh?}1i0Re;n-cduW&KGQQtAv zN!Qm!^|;8rD<*?cAC5JMc+0a!R@ZjXlf2EcsAqoAypeZ6#PiNI$QhQ@Wd2I)Lkx3+ z;rY+pOQlLGR|@LZh`$`Qb2@DpR??VOpY{0#N8630+}4rzLnr3fy1rgHBAAojq_-0N zHlc?NX8O@RZ&f#=_OOFgmE*oj9f8ZPGl%(ZU8R#rNbG1oEBrwJV@;o;7IYE+fD%+W70sUn%-)| zeBqv!Uc_1{`)zeqrKL=^qxYF)3)ngHgG>sqJDpU`5UQC-T5Z!YJ+Z}F+E->X_&6zs zv6;5_lt|jQ^6}$iiyw~locer6rw)jS8s$d1$*^qpKz+uzL3mf zt@-M5;(WhekrVrwh^e9{GnTnm=SQ7)SypRHWWA&`I&nODd@_HM>XfrNapS8{MpCzr z!Kxn8-(n>{zS?!EJ(9`fS%g#zB_j!seoAQj-+#aMkkGv0og7?)$$zo2qS7pC^J^kEdkiD2eJTR*5YlM^6buMpvc6UB=nl+f%W@o;dJL{E)lDj0{ zsmMqt!r$8I4mv!N;S;vG@@!t_DE*N%Zj%mM)%4Qfg}$!O$GKdkAYV1=TPJ5%f-ZFM zCq8|gI%T_G>OuV_3IF{hCgSzaG}B#@YTn1v7)(1W#Z&a%^E5m0X3@HGVd(7hP0`F4 z<>EU!LiN%)``%0`-i7(omSk{qrYCG@2d^J43*XV7dD{j~dhUxQc?wiADOUtw4e~j> zk6&TSNn&rUO=QR;qOOtkmk00DZKm#bEV!lv_l*76c0_~PGB;$vrd<4qdTm=+gNM5( z54ZP=)~MO0Ik$pPlO3I#fr{D;p-qbQ>wKSY0QDamiAyq`>(!q%^h^k!oOkCV_L4a* z$g+O4@R^~DuejWj(3t1*y9b_1(vOdTgJQBNvrqx)73$+D3s+No_xbZ0uc>N(l5D!! z7SKhOee9*Pc6)Fs2H8qITJ zjuENTs;z!}lv$SLzSoaM3{IPQ9qKADx|~V+<3pw@XF}j=?CoIhilo;e>4Uw~aJusi zfOtv%#@Kfi-to_s_ok#}g1(sY%4RfaP+C6^Yr9;Q>rSPfSTpZr9l*XRVBc@2VI{Om zo6gj6gRico`}0b2%^|jBpKqng*T>`n4(r4=_TJ<-c4jKLbMaU`M7m+)27=41bYie) zk3vyD4X4e)&gqV#>NI5%^6`Z$BGGwK*X=ugz+~de#@w3Z>?dypra7$k-`~HgE?dEE zZXI3dVUXe6uzBiw6l10I{8iWCz!_oJu;}o7RBnUtOvi-|k|w^yk?EJTAzG_?tG^_0XSNw$vT?kyiG(<(6g< z|AcnnTwm0_R;{QOf8ExR8)gq{?;d*r5AYL})nph_PNuN9$m ztN-1^y!0v719Zt$DThbj^tRHnneFOh94h4t41M;9>+^enf^weGDYQQ-3)0x)m3{Gg zsqkB_scYhUGY5H&dGSOuyX7BCowRvyI*sMgC*x_WD&Rf0@DkTRCF5wMuT)LikIfvz z{)a18b-|z3ompt_J@hUivu}Uz^y$=-X*05Hc&3h@{o^|g&T4fxLhL#nvnt0&Q_dVz zW!9;aO4N<{mL~A@<(AW;K4s1!ogd$W?l&9P4fem&co}u2tyeioDJLvY@#Blf>kki~ zwI?6@;`X3*Nj|>Wslp=gIc00sJ;$NdemJ;DhMFxQUq-o5z|~z^g--s8;F(Vb!Ye->bl;@QxZSN=;8A@x zZ#v#>*Vp3$c|mfX(e9hyKiQvcVGaBMtGr)wI`dPen~mhPTQvF~=Nxo>nhSKhO$|0p z`(BpBs36V&+|SbO?Q0cPEne`u)zjzuy({7E)&b%k%CkXJB#E@9@0Z5ERv0LfLij{I zpQQ-!wK5()W3o$BSGdZ-n$7+~zvBsU$K*nFcb(0n5up6bQqV7>@$7{@lo*1z1w$QvdgXK^>f{xp_PUXxereR(kbW} zFUj_>9Da0Q?+yK!>jDqAq;#T3^}9v0cQfWIzf)-9c&6YMUM3q&FF?9!c0QxO@+|4m z6jIkmcXvF7pT4WWZ7_PP;5+4H9@&qx!SDU4t6kmlB3g1)cv-r`dO!K40oNl%`TE{! zidH@9S=?n|AUYL!^|8#}=uFD1PkGb!SS0Jo8Td7S&@_A;2t81H0B{9-Csb$<51^38 z*B;OwC{5*iv44MsUyqOvV^DOR0#8^(+bJVvyE{e(T80N6n3YN;xhV|QX59TS-#(Qc z(Rz_=REOr1knkg6(>c>m)_V7M2FJM%m|xMk~3nddBaj*$yY&_I=B9=RW{;~E$ z)!PmyQ{V-?Q9X^7MBs70Dp;k~rJZ7j7c#@?){BhF*y`t$)M>rJ_XOCk?-GZm!v?u3(0pIeFL$nr}F+YPl-L*)N3IM27d zlS7zvLJK;-!!KA?qxR$@`IE*cf6P0my zV|KkmNsp5|Hsc4~5#nQyABK86-~4>mtJU4( zSdXcnQBdk-y$!k0EUXzH{O0l8^P!B{<2}pdLWQ}@rwm(f8J12PCVpacd{Us8^JM?N zMfWd1^xlnl+XfyYlSwsaxld`&H=SAc+EDxTtQpb3vh#PcAcbj_v~R8FbdF}$Fg+3u zP}h9-#HhC;ul38#eC8!nJI?QcGSGoTt#cl4pIz;nTbn*3Z^NG|Zj~7MJ)5a0@eY3O$}O<(bNpE*<7(ck8ZRHp7^eGK zhDN^3kdzntn)G7vYEX!mk?xNe;jnZ~3zjDesw2~80XRp5l(u?FSO^Wf{I#>%zdWO_g@vXQIO@iegr6c z*!_*aLx?YTqGLv^TzWv9yFKRof@{#&V#(p*#PNlPzSOqx@eHg`V!`Z0Y}c+_$=Y=H)28gmL@H!VhsC4HakC z=)6~6*H6B#G+@42kT$*8%yRH1AaPwjOyZQ}oonlzV}=)MUw$IzxWo2FnZ!PO;(PQH zSO&4pa2hnnRZHaRx#aqfdO=%}vzph1_cQMm>D1P4ePqObF_cL3{pLAAt$f#BOBl?3 zhj=FUjlSqNPlY#kRqZ=TnlkIGC(wKFdXeFrm{eTGvmjO89E)tbq<=8w_9iqX=NxBIxsV}`sqbEyYx zwR{f}lrLG5MP7C9c&R<{Qv02C9PF;c2lrzyzbFjJ9fVR2)2JB;hbbRCWTD5d4t+SK zvT7FYa<_I?)H|mZfR?ieEL3Sy6%Lt<)F_3jD5_O*-q&A>>1l8Cj2A2)yBkPztxVnb z-kgo7*V&5CQL<_?fVd1#+Jn?PHph?Fp1oa?l2+|$7EoCt=3B4z-Numl^RZgqXE$#` zP(hiQ&6c=}7kdUpMwd3Nxj9SckTp>X-(;^rH}#|DQL z-#F)TrnO4lZ*ZvT`aF9oh3Am$I-8>T?enhD(w`Pp?Z+AYwa&qHZ+kmAzjgSWYG}9X z+v6bSYVo5aUhHm$tdPg6o5O2L-&L4PhNugwS3g}Zy&!m~X_3@@>HU6bmm!H^&7d9E zkHwYg$iyZu1r^r_DV8ZUE(KBkcCnW^xNe_RMAzx%k|M))4>ji~fa z1slPrdgX+(^}*XA-JK?Ts0G@arp^(cj+*stUy{~(Lc@IW?!mHJ7}3)^ z9=D%ZGuqHRNr103EzmcABh%d&I{&Qac?Rprn;ADoyWS*po0lHiJ?(aMG5JL8%^yQa zWMS9ExnTCjGb>*k?a(a$Ds8a+_F2e&gAc7Cd^y-`um^h&@|#56_g zrK$2Kv%A4+CXcO#-s^2RZB-IWRuOlSne3f1YVe|(*K7~8C7<{FLUSc2v1jVdg$1QJ zHV(fYE*8ywt?RDu1S?a1C>`AsH~Fgb_>#x)hbwt`^hfBITvtSPe`h|D^o7T1HB;}q z+bW_@u!VjY%H0v(KYGZ9==Sp`>!FW4(z}NEZ(iNWV}8Z%qg^ixb+GAP^(m{Qjk_Yb zGv(9Dt!ydt(B-?KpOmD?1q%aRMoV{_M!P+5=KU@t^{PNj*mWmD;u7b3O(OnR>SETt zy=ls^nTLydis9U!G@jV^1~p#pYJ48Y3QO}S6M#~Gi&MI3HaXP%Rz=pK>A~*o*W$zz zb`wnT?aWt>~XU3H%jF3+%k$T$OK%MRPgpRtgO6Kdb$39Whl#;5wek;(Sy4K>LIB|rK!$!V=UZXRDHD?;#~Aq?rd13s}dWFPhY#7!f~?PZTDd|ccn{V zH$4{LNO|#?F7hO|=J)Pr@mMWuJ=lL)lr!4NJhe!&(PS5m&NKV1xnACjReJ{M^V^tO zRzz9sZq+OkMK#M^_mFCiU{+?YvKcI3vtIA%E%r*d6d!5(4KC-MT`qpIB1q97U)<-y zwHql7NiS{u{2f-q>tXK`bb8D^!;YWNlA+x{HkZ`ZAkOzK#{aFb&6I{) zSwF`ft(lxNL&qCVb$)Fv?dbi^F}%qr=AdQQEHhhjvwQQ!e5|sU#@LM$JXUI_8=otC zPg&CM&M2Ltb@6Z)-n%?~^wg5<3|wEGc9Q*&YiG}M`>2$8?JF4_kPK0Wl14e1f$O=Q z{>_mjIYS!GOA+}OJm!>KKS1V+-d7^32X6^~aP}>Xt5Av40JZ_Do=EY#_sm8=-` z9pI0izOY+yWUJMG8RAr_ICOTcO3hX-i)Fk| zoK~q`!t2szWZoU%RKMSeq945ZCrkqG(`~fpeVr---XUgHB91oZUJ3v1daQc9h`qeu zaI%2$lTOM#r^38G!(M)ya&8LJMXo|aN}m@}!c5Rc8fmuEka7b**UqnAT=7R<55C&X zCL{rr$(gVw`Ut$Z^W@me_;0)N>3++ z_4awmrcp+`k|1RM4cj!a-2F!G(**}#@g_EV1T75}PQ(f6MTxDOUKy~bEib;pSRpa; ztcbv(d@=kDLw+hA z&N+&2~;MI3iITaX^C3GoRyafqQRx?|s2Xhe z&A3#G=X;e!@K9&GYg~}d>>W#3P*S2qKex(7SrxZ?8@Vqys(sS0IT%rSv)#E-^>i$o zm(e3^{VhKOf1P(uR#`(ki@wJB+f5mH(-bH7hRd;0mGi1Ut?#{5e@kQTwaCiHh~WI- ztx@S4Dc?r2t~t%?K9-Il@ytppycXX1E?YFI_a$)sD6ntZAq_C&K&T0Z+tyRwPj~z{ z9V8)|@JPk#Ho!7^%)q!h>HtgSf!^`nB-s-W+h%QB?N?IG-*r0bsgG#u`v%>up1t4y zF}6?1joV7jzp=DU6Gk?__|{sCgYm!$FNIR;{KuEN$p`@#c^;)XR?c&&c{fU83{KHM zFA93Of31Zl)l$%PUmDw$I+^qrYzM=nAA4|fe=nz*@b4LE;lZIKkq_x4m|7Yd927LG3tuX!0BlGe;r>L)7#SoQZFFeZEPex`VZRWA_h^ zz7Ra0frDdz1`hqv0tA5mFah|13lT~5=LxVsS|Bch{K&yS`e?yqr`Ut@KI-!=jV=ef$y5enIFz_6o@e@9eeo%q<C2 zE0&BY;vx_I+!nFEio%P%_w{I^Uwm1JqmXZSM?;rFT4y^{Z#!IHN3;t7%0h_a_|M4G zOwdFUt!AI6Wgu2bn7Pd@yRPqHzjx=S5%XB0>Kr17@WF$E>(U(hev#ROF&3n^cE75* zHPsyxVtNJuJq3vc z=@1Lu#6?xB`4{~b&)B+&seZ^x?rqT*+ga01wq?3|@>1woie27~o^MwTk(sf&j7xZ)m=34RaqU!%_OVRKR#XUgYegoj1o&&rkvmT;^wVx zu6QCuHyfpezFEty^IO#G?cDS%Omski#+O^r#Z$iN^KMsrNoA=?dY^RF?E^5 zs#-4ONMCImRjBf_l?}GdwQ~xqhmJoVGgS-`+mz1F4qYJIp=^^&Oux$!kUyoZy5m|B zJ$dM+$tw!#EobVYkZ(>r>DSKpW;zx%EEHa@+~OwMGXFwHv}y8==(4R4)8Ll)8`rwr zdiEmoFI)C^w{%ywfOX@ft*b*@vez!A-1syN+nWBd#kjM)wr_oNhlR#?#0yG!uFFZp zU#hc`TaILZh*TUjo7&>|@alY#qQ0s``9^TG$CI7CER;Z~HM?a<_gsVfrk1Un2Wbbz z<%?T^4~V$GSXpf`*J{3NWnLSTw-#8DCapj8EmP^x_JN~Vo-qq-AqK~W^{w$%5m$@k2;zVh1dA44sT3eizktPz4a(BkBE*{ ztG?P5APS#V*euxsF6XdNMlgSMe`wLLp0aUrMgKOmBW|dcsqN_gmwCe3i_cV3d(>=P zuRC~_ZCJ)1Gk=1(=)bB#q&=;fJo2m{N;WsjlE2R(f8s^?9eKMKg$+DaVy7juxZ0o0 z0TN@uf``s>lez0w`AaJGK-3R>JKD1lDQHw48*L&m@nF~em3X0oCT`nuKJmL?f^2U|of3U7rlgjve3vvkEc01CPc&w> z&b?l0zqf~_CFGm0`SYPt#lDPh@MF6j_Bw9(O3Yp4To!4QecS4&`gP{6I@jG~K9v-I z-3A6D-JGe^USS`4%TKa@{;=m#Z6|%x=;_B93?f|VeRrx0nIPE;J=MP1 zaxG^f=Z{4V4hzY$C;A;rjiynHxOpWm2mauV{I@ncTIY9d78Q*@P82QWaI2=nPbWNA z*EHO{amshpEH8DASz?3Ef=iEgp@ZoP+HS?t}X%!!^>=2-HgAT3r5jRwZ zzfE>nj%^`cVu|8-;=-Y0!)`{r!(kKBbF*pMzj+`tDp&Px&doi-Ewq`PmQ=8>aXC?o9HTY0vP$;?3C zW2~wrnk}XGlKbr0Y(CpvYSQLxIoj9cR}p9GjJRREdXDWY^}DQQ9iZ1F&TEJg=3t%M z8819wr*nv|(_+HgaFAcv5n5uAzo&?D@8uJ6*{2<1c#TAYoN5tfj&CHt8@~IX65a&S z%suqt@~KNh8oh@<4pA0e*I(;cqUkydGvzPbWnYtKepgPyX2xoL=MVj&Ykp&uryQ5U z4qte(nHhN~NMnwZAw})jr$grb0yl1$bA9cn$VP@uV8k87t*680`Atn3KpxS}i_fAWdz6Q`%?lRUJqC&n$LSL-X( z%QH&~6o|w#UimRAxlFTPnJsUA8KKmuqugjj4t;+zF=5xPt1C5#m^Vr$PZ`w1slr^! zMkakf`44ev$Qv$jl{dQSrpdB?$Rti`Zj`7zS9XW~MX5NS=U$qp@lsd2-6IqBQkHd> z2i;!oui7amGkbY$v+blucub9C^jPJA+nh-nvrSfp3+MU|RSZ?uq#Mrn$|OEaU;Or! zwXuO$d;4*J!P!4(=5n7DQ_XO z?%TI@6T<-Vv5}is?yb*T`8BP?gmyMA41cI->e*$=o_hh&JR<}}R8KVD^6XP+Ug1p9 z>m6CqUN4ytjV-R~ckFmz8jmQ*@b%bytlT}z)VQj6^$ZnvWub0yu$8Y~bzGpVD)IAM z(%QpW2>v@Nw{I<4MAsgV3VQ0g!$cx$pvsqTe%NlmwrBa9_V_A+o*!S1c`WX*(mnC@ zYO0^3ZhFqpXrhNR74LVyX#NkQ3Qccqmb#=R)*k5Y{4%VVaMjNxT6Niw_WF!=&i!Sk zH6z{*7pgm-Sxa)83;SZ0m^7X%+3d1PR@aPtcf|6l=#%WF&+vPnD?A^wJe}XS=fxXH zhm2glldy#2{g29HUsHS9=nqJ>DL1)1eKC`k2aj7?xPCS8tHvGLxHjSwsj9Cm>=JjG z*i3SG>O8*F#C)Lq?6=@Yh=L8#1Q&s5nfj#0`DAM!#(Qddn~mOT<_Gg4caLgUd^hVE zt=oN5w%E<~DebqnylQPRvB$uVPc**?Pc1l=nftWZ7BYS}=)KV`3aTKsC#vk{`Zg`K zPNrU=cMkWwb)mO8T6|-Qludp>`>~MgASadG6!E){*M%xOYVXy#x5Knt`=e*a<~fJ< zc*|AU1m%fNj%b`VmiSg_Qc7I;a?H3mmGXEJB;rT~9? zwtJ7}A(3GX*B_m{tmy)AonhG*kL@rk;ClH~yA(2#z-3maGe4UdUU=o=>(k^*6iwl; z`3*ClDb4CQoYo7J5Y@Qzi9DP^rb3ohK9I5MP&28&t>p9XmDKk}0O$Re0|XaSIqhC5 ze>o|4w@Sl*i8dmNnE9+@(cE&A*lm5YbD{~B{om!qdMJgxD5f@;)IJ2p>!ukujC_9V zd~{QeF=*Z}Slo1jQS1nFzy5fJ>f*TCocz@a;YacCs&`J-3gdK~vU<4<=jo)NS>pbl=G^(Yx1XsT zhgET2=2~u1t>b(j0&ACMW_gm~>G6LG`|7AFpS5j5kWK*+q(eervp0LQN$CbDX$0w( z?h=rY?gr`ZloSzAq(P*myOECX@qFjJ>-^a7TJN*?hkMQJnQN|l?z!iQxu~trMkgdm zq@bFApsJaHE2T=R^)xXIE*9-bdxhePKah*#8DCds*;g{ewC%=wI_j06duJc>;eOL{ zNc-kGzMKl~6h+=z^gUNO$K&iNHZP#|{p-WzKH1Fb;aoc7i z%1G~`ycFI#b#X-40d$241c?GywoVuLQVe& z3$y+4FWbA0Y~%`=ZoLs~yjQT4{z>(~}o?O9{` z2Z|bOZ$Cvf%?yoLo>=0-4h!uAO3o`O9J^1cbW!HqnE1lM`l<`;LNcBn$SI?m=6fd; zzg&{_UXv!Pb=2i;Frb_LeBO;)>0?LJ*~)6wU}Qbzl<`p9Y;3@vvnnxuv-;TJno+(| z+J|d2!x(3O)O90eOvAiYy3*KN@8o>A0b_5|IwWZ6E#+OGxt0UZ9?5QOfDX0$fLZF+ zaC(%VF=#QWZ5i{yaL*g$oc=8kMSE`f0PB_z>1ooRJmwpHidY%kJv5hEJeMM(^xlem z*4GS9BU2<~imY!lHOQ|``kwzepI>V+YCP`yP%%GI!#<;)NW4NlPHX^`TlZuzF~ogG z-*EMe6ThoshvX|Ib+0$h?yan1@h{3Gq2=h!I^vA}@8NN4bm{F{V-%K3WP&4(vIQOnrz^>P?+|oCfht?uTBfHeUWuQ8x?xjKpL+P@o=_q!d z$24}#G!$Ni)m_V8Lf-;JriEB z4$_HWr$|kgEE9_r+QrdHc#VasI7zQPnfGRgG1N8HXD~QoA|$2*Ad`4d;`DT~gh~Du z1@qE(wtXxt8EyUrHnQovfnU9k`Ys#3qSeKv1lL@txG8dKcQc@Trb?aP(o4lM9tl`q ztUas28_2TzH8tiMIkXTd+N7FE%~n|jWm_P(kbVZ2=)e{jdHx1#+0>|S4_eLPw;enHvv)@T-2hrHg#p{Hrnl%!m_e2F_mNf{TkrpFI; zQ-Wl)89vL{Ti|&vFA^xgv8U+;la{KhL5-|mJ?*|N!F71IS~p8*F?_@fLv>k>1|m5K z2LiJ{B~CTP2;vQ`l-F?$ri3&k1Y;eyte1|4*H`sGYkN=vF!J}#Zz_Y8WA>f*&}h=$ z%e1jmNgi;!AE~ z0TGdKtoWbBeb&Sn6y>7ECwv1U4_G7|5A%4N$G8)ZP``XuX2;*fP7W}My=0B3V?c{) zH|@Fj#s#i_(rS}GMMn=>-AKKS>Gp5*(T}^yU4F%7xY0!wy7Ko6I@EIs3ycA!XK`l|@ z?4BQZ_@0x|YJ`r{I#k$d*+;{Y`RDk+VM5!1m3u$7%No#Tx6~>Q}9&8_K0z+pLYL35_bllVY5?FRel^iuf?Mke-^NE@LZ? z9`9Bn@p+0-Ht0Lsmy!pK&HQkIMB8>vk@D$uu`KdUXy4}X(>3G@q#S)egbhGo>sW6dYK zx7Trs40ElIe@55IwTS=Fj7ca6H)}#K#IIXsb>7tDIFfn$GpRyYYxQU{mYHZ4Y39u* z8~t573EQUByz4|uY5TZH+}=VNVZcIa){J2#@r+Bq zzpB4k)wn?^Z?fWtyLG@+w(LyHFqRvqlE<(57>u_%@8~M(z1>jXi9W?(cqAou(dQ|7 zgq6eLV@xgZ_U$qa_?!lw7SmFOp%uwDI~w3Tk?h#SJ2FvsCKr6!ES|h_4CZ5@5zNRq zEx1OqiEoY2)6iQ8!33$Tm}{ zCMauXu7wKqz)$-ok1gvI`kkD;CDQZ~&n}3s^PHBo80&ru zy8J#)j0>E2=r+~R%45$ETwR)W>a>XD9vL;3sy<1KDs%0odAQuEf}7dGS*F#UL04WG z7iXR*pXc>yN<}yOBHb1en3+nuz9H3Yt#x6{ zFS@ZexA(i!3O3mV?Rz6fvdMe?^1B;9x0KazspEKhRZU&SImptE z+J5yXB#F>uT7B~E6;tI;gQZQ<9`d-xn960v?+eLZqRkjj`;=)%(7=a^bq0=sv1lW5HeHKYr-W z!9BOB~B2_4xTm2TXSZX%Lu-EmqiCdaq$5PO%VGz07yALg=KZ(cmn! z?rC8@nV`agmU&)G6)mvoaq*xo~ zEB~I5X*HGAf_lmPlfc{RkEmV#KPE}^GN zZDEr2SF7pU{;%&oHUC-D7TY(iP_J2kpe*b6gPxjr2!AGYVzsprTuW4ulx@b^7^&vF zqzmDkvO8~yCy26sMg7BBMc%V)JwEZ;*n{Xr*>sfM>QkD!KR>;xpEMBk+OEl7pc4@r z)C51f*1~MRNjI`%66Y~ka-X`&0H+O0$%{x~u`F^@hy2hR?P=4S_%SW-l}&c}j?)(} z_6$PeG!Fs4q@T6E^4AF7G@zks9#v{y+%dw4JTP;4MV)9`WbyK6O#=>{;PUgA*xkXO zc@vu$Z$eJ%Gk=!FuIq6kt8#w8mmt(q*%$tKO;>Ev?`x95&0&aaOh&2rLZ6PcxP&E1 z#;lz=hlj=6==*z9!jrNsT0-4UO&+{SX{mY7(qd7CeWnQ?y33$3w%cSu+O6a(OH@uG z+xtk!T-R&$ho;R8NjrOO(p$w9Y8eU(~eMzKf zFA9eSvGdq)doW#c%`2(HWUKkrEEC_VZl!@*Rx!vgZMdw|cAC&KrQj=UG3;iiHv|pC zk{X^TO)qoNCiI*LX+;UDoK(^nnqOEwg@xJl(A^r=H~4IbxY*BJ<(1}t*HgvLA5Y%( z?Dk_bto)-WVE8849p#N`zIYDW^s>zaJ=#%+nH8iJcW3#TYpu$gid7+FdmjoLCayiN z<+1PMyeg>|%(;=MMsKSd`j3+Iv?-iv>reP?zGZdR6S0IdM^`--?8g)k?;J0*vg2Cy z!41|2QPKW(n@> zYRr>vQ)OcU)9-E^znzSr9!||}Yfq;vmi6d;{+cUK^nJQt13kP1hRJNlDtpWIY=%0U zJ&dI7(^0Y541-@TN&9ZGhfrtQV$aQk@o7a$FfK#Z73;c}qx1r?^!ZNB$1Plzp^i5$ z3(4OdVRF`{zGnyMZExle46=S7@&_NpX=fzrg*_Gh$o3lNTHv_}uC4Aiv%%oy$FQ+F zf3-$q&O5F4j?Weg9g+}f#`CT)m zUyW@O1!yy^F2cswa_9a~rs-sl)}$=U zNK9faL0N6q0wdT8b06dRxg-Z*6BN;F;!yhsE>BO@)vB(vSbd88GxJH8gE$h4wPA)s zM7wX?ht$Exb|;&z>2`CqPhrXJ(EUBpG76^_ZtuGN7i3y`T@a;gBJu1`ybtRI8ZOEe zah@o(-4FPV9q{`SNZQRNW6%%hHGK4zi}E`b&Mbn;$EkB8``-gH(o=sZ`;900ytDV-^M$lE?%l+t z%7OC|0y{3nOM^3QV%Oc>mX$A~tGw{qDAK@*V6-v3EnO__4=Nf`*%byJ2l+4j8?;P8-t}$-OL?cC8m;TM2C|__{`>Bci0PZx~Gd<}C!lttqor1JN zODrE}J~GsBc#!lc$_IF6&g{NbE~Vjsy=K<5xqRfB{KNmIg< zN7KEd%7uwkuacHpOqUw3@ukk_V*j^irGGz)|9gglfWUvguLeHDhH?St75vX<*u30- zkJ$LvN7?`Vtdy4*3Y7XkpOx05si`fHL2*Kyk{|(2gTr%k(n)=Pzr!9MemeRw-p|iT zCMi(mUEt_k1kLj~&dcq)v+_Uf+Z8-#LY~5&S#3k^r{w6`3O0Bwa#D^Vt@Jp9pTA-f zq4yP8Sh3mLqrRY_QX~kW2=Etqh}9T;z1wLp9MIGx`qZ%NE~l^QMd!i8H1@~0oobGO z$nkSK$hrbZ96p8|$c6+Mowz8NHxf=xgy?Z}1mp zKbd6qn|g=V$;iOadCN~pvx;QzukB}u#KSdG0bqBy^6arh*<>&t_Aov>z@wQ7*a9f^lW@OYMFS;M>LC0WSV zFFl^fcYLq;lXD;bShq7r3)0fQvFsilz%0TJ;A;_yH@;&KOy;_+))M=TBt(G0(DxP1 zj~A6DlN8$dT5iJ)F5<)Qn@0WV;Mm^52}M&7f^rz3EkdRLSg69l6$5E+3H|W+qH(zc zW0|&=aU2DYD_B#^@bR6oFPB)^Z6`j_TYnFfa1ng06%;=oL0?}%UxJi;`XBS#)R+6J z=(5Gd(rl&GHxtJT`W6;aN`Cmk(KLPmX;IjnXiLlJ7zdar*SBfX#5gy#1jEABvRrIZ ze#6v%gsjJ3^R(QtpsooA2cjJ;=x#;gZv`PG-x0Q8hTt0?9(?(8$9}Oud*>^Am#})r zaeDV$W`1D)7UqAU)N>b%5rZ z&*w2FOY)z9GWabYMp3E2IPsfe%pjd#*B?`bcRqD64N_qRnDc!S_0{9Xr==ObJE!XT z(Y3coXYU=oW=FPS7Jvw!V+DxT#!*=|o2U9XsWUgw=!eL2b6!BkSo$n+&$}qLDsfkovbVaJWYYPl5T)jZoAIPSt_+ zO>un5oHy>st4Y)oIy0qVU^SU%K_@O6^}iL?+W08EnDgZKUp-wC`}szmZlS19Se?+O zty}t_s9R@mb3dbZKW8fhPsIg)o;MFaV7Wi^MCJQY&a16~^`+TXHircn<^zxhO+4+$ ztA0bTLnZ}NZn`qEHn-*64N1e~ii6B42SNz6o9`VB(PSr%Qo)oKi zWf%htS*gJiO+Gb7$O|v~bqdyP@dha;zUD?_oV(3BHb@(jDLeQ=Tsxm1e0C90f_X+} zRVwp@Zo`NqLNAWs`|h&Ki<%iV#uT2+Y77TIs|BuiW5pea3FN)qzYL_R>avWE8kcgU z?d8J1sn}-nXtT_Uf*F=nEh5Zu4vVsOw}wC8{@{4@;WWVS59R_r_cF~s`PJ&w1Cvvw z(_=Q#I1FA=*P&^53-p}>}oi;pSFYY+T;0<3R&8^|@;jq`_W_P4Y64xh$kjHs`5w~1%nh<*Oqu6S( zxBXV%$DXTDtUSoI8jp{5<$w{RqQs$(#|RG(xw{u@Mo@o;4!hIANcgCPk1|n>5X>pM zD{KU%qv!BaoP4kPn@^PXDnZ=PXiD=&Kr2*BbVA~~IDRTrh?Z*yMfGiIfa_NbnZ{X~x1&p+<1Q$nP9F;+`XuhisWz3!(|~*Yvt; zUyIx<_hIT9)|R)%jV?&zCMcoBvU@jD>=4= z+bqQ!x9v)5(Vhk)3RcPG5`E9#zwX#7&dF68X7y77&1)7fy5HUF0a)Kw%9PpZK&vyB zbQO~2E&Ih6BvIiRw-D7*Xn+I7mSmR6#=u2U>5gSyvofC3!AekS9MXqip2)jq9t@(% zN6P=gV6X5NF~Rfo?d>#qxs+O@vdA&((4Dgg-e0ZiZCTLB`)I=MzVCIH!xx6ypDlDI zULRKJW0@*hT>BMVDohrfH49IOL z=rg!NV+(H$jKiCA4fEumvaJQ-fwd`>f8e_d4DOEW*76p!m1jT4R-R?Rq1a_ws`??4 z$(8}j6iZGyq4{i!9r9s=R%~FUA((@uJ;QN#-AEZ3qRn2C^v#V#>Q&W89oPqXbW@~; z-WtZvPe0=H$x!S`Hk)Q0%G(yPvsQ`;o_Wh#iA>BAbFTbidxi1xtI?8hF56rAs6~bm z5|iNN{%Xp%BgOj|vGap8OD1WQNPI*@Bd55-C;2H<%dNq!@_Wl(3{QRM>Mq@ltW+i9 zJbXo{zXy|X>zu2L$nAt=(UBkGd5(5yA9@D-QGK2h_SnY$(X;+7KFa9kUxg-}O7cHS z_Op5+IrzQuKV-4;f?w4Z$!l-ZGh-WCxf$e_q16R?&3%ZFdyc6w6XG>E%~V1=JqCSc zF!edEudQm_zShT2h`4AcU=Gsx#9tygiMjqDjEMvj8zZ5 zDL#p(q>sR(_BIoG)X63$Af^FWAUh#hmc7?i$k@J+} zkPl&#HU_Ealyc$=m2uz^ST01i+VpsiUhDZErHAYg>AMq)v$%JM_oOWeZa=Oyn*iOp|!ad^%_pW9FW1vj=z}CzQ7ue zeBSdx7K`Ni+h`WW@2u7qxuD08rlI#$w~{5)hRgP$IWkIDQOP7}2ERi2p3&$@{VIPs zeDPG$plf)am)UzL%2J3qD@{fOW-pTWMCbm?JCVs`D5bx+Y77%34jX{G@CKKyqtSp%C z;{ECL##oT;4P@i=E zxw)J6XHO;8V-$Qs^h42qfBGgZ*s>xVQ1H;KF zknq;V5flD4@HmahvkLcN>*Lbn@I?hz(I1b9e2mv!nnH%m_F{*7*xJutVXn`-T1z%R zBvHE3=^emu_0{xIsn2jzwjnlWnX<20n`LlnlCObtW<8a1UnsVTiSAkN+0b~|8auQ` zvG<&jy$rJHKqiOw#GE`8&m*XL9dMbNuX!nR?-bGePnBa7sI+fNT+KGV>u*_kMGE0{ z$?{2lMu9+j(_#H?qMOX$^9xV~8J&KuUDHx!tqj}jzu)O6+m*O8G&aK*72Iu5??p07HDTJ%PXmVeyQ%Z z_O+LJ&dVuQV=SpK}R=>i6Bd0#bv5xQqX&?6^VAL^gaR;Gnlk)sznVn_@KN^ zAoaVLMg72s4?VCa!J6LZ4+-ANdf*r#UvD@t;l!~@JlrxuaUm3z>}K{zH>MZZ#292v zZ$D`iM%`L`wOASJQ##EPo#L-a%O~^ep`pQAXWWx}5FQA4l9;75`d)YR(&y98h& z_dnorLU!H5eYYB))Rb6LrGUhT>lG(G?-={@ceC^DuuHiPc~Cl-ytD9lPwY5iQ6j$_ zlSPy2;^8_<*<`N!Y!Zk!ku`>hm1f9g%s(txV}CSehrCb2Y}~JQi)XH1UdgV{8h_cI zQ;~A0Ycl1m`p4jm>Xm6Y%0lC?tKNkYg+2}Sb9iv#iAnSuXY9*v6CVwRt&=wUHSDO1 zP0P=%LlE5N;sei_6^B^*p*<9W-ugUz{Fe}lqW8D+3_uJ~2y z+fqP`ThHolrJX>y*uZ16oB^S1!o3`p+>@Z>t{#rZ&nxydC5#fbj|A*ob^WAFURiRt zc<`~Y3nkL@620{Coh0$&O7+LhS!=Zm<$Xj1opZN!dh3Zc&|8eZ!u% z?*wOE-Aj&coTy9}tZ@6ta_u4E*zBk?R>c<|(UUGgo>#2Os$W>0NUnt&M;Htdz>CGs zEG!Vs6>CvaP!SHbQOwKCtAbZGeyB$^iaM`%yJ2*`|S7Ap2Th^({r{dT# zxgLE7#H+DY406vc2bSle5(OeS=ci*$6pFqw_qz4;tv=ot63X|ho6XiP4}HCWHs+U* zc3c_~DW-rCPw<5K#q_rgC3Ti0+f4@IvIE`92DQy3q8P@vWpWRHPT3nyv5JVI>`syC zamo&ls7ytY@9bHYq_Mqt5|zER-T38($LVIJ#zO?^ew*8aNs{^awp-XW%d3$9Wmn() zWxASgsBm|mfP}&N;JNc{_Ycxx=|@--z0pQ}vN?^Xbj4 zp>d0~wZ2OB$;rJ$UvCAWGw9OgpO-g_ZG9q6!9I%RG)2_Yyo(y&pJT>SuE$DNXV^w! zaN@9$C({fyW20DOOz=c+L^>EEb&{NV6-1~)&r>?D#``TiiuP{4v43MvXV^Xc%^==% zhSccw)3}v&zaD&%iC1=}AjZog?ECrDUN@p)*R1!tY^c)ARH5s*#(AEG{=hT)gw^e1 z4re2-&G!Ym*zc)}^H_mB#d!KY9-G^aaipVbp6V*lpS11dalY`8DNOo_qGt(mYy=P~ z+q_MI^(#<|sDb}JS@-3ldiPJkiD4vM{C6+gaNG=~z``bn{;7Gvve{wT!#lE!L2wUIdgMh-Hs`d;h;trgEmBbQWt-QrbE zeRakG?7G5p+C0IfaK&b|piR3n({+oo;WDUDU41uGuUV%p^vSR4mr#f1`On($x2G4Q zF|R7RNU}6TFmVVsB)+{pQ+|Xdn_;xa2!+<_6m#QmGy1gDpVbdO^_Zy}ght<%HBVSr zqfpFKaVtG&5HI zQzBA1CaC(5}0EK&+y$_Cr}>S-AN;d506X9~We?)P(BG%g!s^ z?dmY0^8BX2O+QgA$JNMy-{`rLCOACZX;hv^j6ragg#(YCXBN#1CM}LN=f}>kb~ZSa zzcDXV)vgEa%IaULt|aY0V0$T0+#Lg7ruGJ})3Z@Y^%JZ zksV#2UTURAN$m=rp~0!%3au@%Vse0kEPqvA6^ozZvojOCd*}VeU~o{`NbYBmT#8+q-iy98f zJ^NWzgqZKEPkLmRv%h0bEh;FCXqSJp#9-wTw6u6yzF7 zxR<95j%6J79o>DQPP!-x(YMgvdqs# zZCoXa31!rh-W)4`*)P;*7%Xv{>r%i@U3EHHh%$4fIHck_U{){Xtfh%k z@vxrH6Ygp8L}w+9V*a)-y&`^t4PRKp`;AC9%csS&qNlKoCTbWpny8fdurs{UFAN|C zvjdV(-qf$w2&W!Pe2%%;aAB@DKWyoIYW4MP=$>jBSoJ6GJ0=R-k)!BF3e-a&B3nuJkP6S6O%l&I^2!wS??@~&F>+5pZ zbUX5Czerwj@Z%-C>|KNIjfEBea9eGpxd9*l7Euz!_IRv8PvTnpMD79pDfD9^$Zz1G zdMQbAK9}Br3-!4>d5u)To(ICOplz?TYrg2s?!I^=&L}{CiD-{ zMuVwSW+I%zj*_Vq;#k+NLb6>rT9@$NPo3CH^AP(BW2}(>_77ns&MZXJ(GPO7Dd`8L z^;Ua54Y0JWOH0K3ZffF4fwMBj$Vo%{l|o#|Jvi?dCv^ui=nmLrQJG4GqA;~HvR20z z4DErK6~%|SbFmC~BpG{GUDw(|JePWZ;#^pd)OHEnwu^{PK^)z!*ta9euIPN#4f%#G zaxTSW{WEYSTSi~>=yIOPP={g%p%Ya`$q~DRDxl0K?l!C3k-W#b6cD{;Zonsk$Moqu zI0l=My!$Y&@SLHgooxTh6D%9ftdi0&i5y|iH)rKhT=Bn?UwsncB?)=@K=@<1gN)Zz zUwNWZz&D1_L{9~a5p(V5?F6Copv`6ZINhA3(x+yqPp8PpTR!_-A-8K4?w7Jk2VEC9 zE)Bw^s{A|G&(6y;i8z%BjsjEqv|19fwb!3Se3DmcsM%->DF8pR?inpYuaboXrK9<< zhMO{Z^CztfP-A1etdEQ9ewp}=7HmLNiGC5vjz7#XtD6xn{$PWKGMu%5=IBtDJm6t^ z&GomX$oY(y5!gnQ+f83btO$P9A(hErj%%>5Tj4 z4Xgy?ikPqKjY%c56d!y9V9T=%`5N!&BPM>m2*hqbN2dwz z&}`*OmM`H;iSy5kH%KT3WyNgE-g=Sy|7=&$Sgr6b=v%P(d|xH(7fd3AWf6*i`-N z`nbKeK?O+mA1>c{n`LE6FOhAQM&d7DsKu10HkAyOrUxZRU_&dqk-guAG0Uj(p(>Y-}*rqJhBjjQagalwXpiqcO;K1;RZ}dFC-K*!Wsqq4Q_8nhl z6&sdUhE}+h>3^=CispAl*ab<0>LrA;Rff=~==kfKPIK4DHD)H3UohT02qtZI{53ea zAIbUW`3lolueq3_(U7DfdkQ6zpyd6lYiuw+017TKe}3gVgtuub@_3f{TfR-G`{$EBGt1hUX=WefG>#_B%z;iU64+Rzjx6<+Ym3dspPYwBMCd4dG zo9Z0pNkOe=wj>Jg_TAO(TRgf7LI>W(fpnEAZ>&>&H68HJBbl9NV;SYbP_}KSbk}#& zZkbf?8e?WQtkwz)2XCLw<@E*D-tOafriNL4B_sdrxA-URg|v6V{!gj0sT3!{`uNU6 zk92GT$LA&Uk){rCDVhb5#^`4UgZ1LI`;J+bv8yT2F8veTXqp#JWX#V})rPOr9XO0Q z6|@f=kEY2yjoNmOQa+6GVHgfeewm-K`jTOA$6ntuBWk=}w(b=`9J%7?Y8B?d87=tp z)8T$Tk?PqMGas@@MRzvFdLC)cOL5RgIlEZSa-9+|S?de;Oxj9g5RT;&>_8oEe9p4d zUPcxE2O!*qj(2BavLGR%iI)0a=3j|zZ{ASJK7B1)l{C?Yea>3YI(-&%$)O&#+`_D^ zmebOifa5)syoAI5xKFZ(ug5X^MLP1$=DI?U+yF_#=_iGkg~9q$=R3K&$wfWSJa)+C z3a2?7Jr7ANpjr>$+;27pzR!AFHZ^9hzt7i&->w%sy#e4^BJ@hx{y7T+E9Q z>OSRQI%@YU)o>*{dr!kgsM589z4Dzk*dko2a>QhqN=BT<&X88Dd)QNlnm+aPKqsmi zpVo6N3Ipkb{-l%%ilvNa7m#)2VH3{6CRfA;6SG+cWW^nX1p#u}X`ViXNJoit)^DaMOv5vpE*7e<~sLCZ|yQ zn?=>5#7M8+uGbxJoA`Mb&IB;%l8!IeyrbU-9!~tK?q>GqMQK~ksc&4Z9llKaf?O)* z+G0JQiZR0kq2Kbx8suI-(5XWfZ!Gw}TsZfr+FZ_V@DW}^-ixg95FOhmlh{kB*G^Yn zTknMS2SKhYu;m}Jr_)p?L&G@yF1A_C%n8WJ{@YJkNR}Sa3!JE-V~>Atjg!^q9D&Oj z7?TIa%)P$s*U8=5vRr+9x_MnLjw?>`XSf8bDaGsEGF$sJjp%{1-hG5QLPQ=0o_J|t1Xic@}+4+L$GF19=qG?>vn>o$yc$ZrLoQrRH)BY_`(;;>W6n9 zVyEiwtVqT@`e~A@teYx5UCi)8^vB27HUnQrqaH|HtmL^mScNO@wFS*0ANvSx!j`Dn zWR7ky$mCmz$}7v>tu)zcQ*>KTwW`3?#|Ju#MufaeCF!rgjXw&so%k1r_68#Oo;~c$*^5gv5iTb>a3Pz%B0B7S=otO zG!Ab$Uc9M)IsN$2$Bm6HCa&f)^jx8j7Pc}Wop{X(KKK2O3*!Ic3~1?@K7^;Njj7v|KkHwR?ZH&dM7V%_M<0U z*H1R3jhA*O8!G|*yR!11!U6^bLtNRjoNo$7V?7HCjTd!#4 z@jc_n-``BapnsZ4Cmit7og<HT^+#*XU%s(ccrNnZ<< zbj4(NNnHNu6F1(}_l%YHv%04T)@1MZ_EYMA>Fnn+xd${xo)~E5yGq#8*7LJj(F)x? zlx<~+`d?m(`L9=EG?m_1nix6J2nYZIEH=QaF@H5AXoQ5YIA1#08U0m4(V^jdDK1H) z_g8%2-y$9Vk(K!SPR(Dr9iW7vnTaEyo?_<==&b;fFme{gjx;)dr94#ryCOF?qC$q50b@PH8(ElQ#|~t}HZGZw&#VArnIymw*2Plmo&6V)?J3cXKc? z#R8PIz#uHH|G5BF4geI5Db0Uu++aX`;_pRc`)?Z@3I}vE{%Pal0)Y{2Tz|#G{#h5s z1qP0s_=k;)2hhzxu)+UosQ_9EBvT{`&R(Hb8I$ydXfC=>GVD08y0tI6&OI zya+hJAYKFX5 z=wJD*dp0;20SAN|f`9|U11J&Qs|(@fLBPTDS10jaIUYch2*Cz}-RA>{LJ0AN8wgXj$AP6|X5I~~tUR^*!3IPX%3yOdP0!CPmfM|~pbAYgmI4(e! z?{fs^fg<<|m zFw6-40=$fkz$Gsnjxf$}Fv8jch672+eLmnoT6WI{;o?T{dk7E{@3#Q~V&Z*&fdDb_ zejC7vTnO_V0xTa0zX!y```G@5CWQWRBd!Y&04_pbz+i;+5&}ldksx3=LLLDD(h>w5 z5FSLl5MG2e0Kx+V{QG0dgSa+Ac!1>WemN)zfj0;gNE#5z!4PsO2#gDnOBfi8&;|^O zunt0CKoWPq4PM0j09XPLd=kRTi^w?~gvdFNiXyZDhaqsz!v*Lu-s=mHVIuf0u#Z9D zoCnAg@3+AN0wee{uxuf4&I5uYaLxndiTC}A2gnl<+Ti9z;2h9xM97DDfPi(M4`7u+ z;G74TNC<5}APAfTt26@Vz)VHp9M~8ka1LZS2%Piqz!7pn!0Qn>2dW})&I3i1zH4r$5LU<77ITVnaM8E+|E(9Eaqu!tAKu-`jheCnf+Wm2X0udj<21DQ+ z*iIwNbLijGAMe)%QZs}$U~ojvdASie=jB1QX9T=J*hRnr2aD1a}AoZ7f)j#@*e61Pku&4#7i$y9M{)PH>mt?i$?P?XYL&-Sd8d zv(C4M{b#S?2Z2ggLDkho-F2m(zC$d{FGvZbqK9Sz&;ZP}i~#KH)Z)5k`ZfkY0FZ%} z5%8}iS^yJ}f&PD%=1%AR`NaTF}tMM%U{8vx%mSF2Am}xsEO;C$zPVm9C~K zw8K=OJ-A$i@i?Q*5MUN|md3^_-9~I21Pm4kM#mTJ?d=T^u1uHl1b7Ysgh|PW$z}s1 z<{j7%63_0&FY8b882U>KXRqo{>&A-cwy+K;Sev(-jA%&1)P+Gv;iSiaw8$@BxN{?c z0;Dm56J{|yZ6NM^mWsc*FG51XlHcIH?y{hP{MszXEgX>o4+}kTYzG1P6~wzW!-pu{ zgWH`3k@O@QAC?ME8uWr28lYW=$*4B#fj1q5;AAR50D2bAx^%Y_@!K7^xdA$W)fgh2od1afpS=>{Kr0a1#G z7X;MuZZvJ7kR}_L%Zn4KiHoa2itKTg%ZUbwz3m=X)3A`_yYCIbP+k{E(HkAyhb)(UzK?b~f`GVMk0Hg>VWP5Fa;GDvvxPjyxA!C3VW0cP# znWP70!P^K78`kngnwo8Q&RGCQ(CX|yWJ=vBQ!p_bAPhL z2Wcq6Km-sIgNGo7zj%4a7QhL4sQ>0}F$dJv6Eyvkf2xtMM{I=sHtg<)UZVzA5!l=@Q4i1Cp>nFj z8-aX+1nRp@#M5ROW0NpPsQ`?yC1ajs|I^t#h%_}(yCC@yu#HPuwD*MJJT~mk=q;Z< z;IL>O8E#$((~l;iWq{o_-{N)P^l1nRVdNj_e?6XvbiVre3bxfQDkdtPEHivOn3z>y zEO*?`Tp4NWwahdK0{6xnC~Hg{S{`7bEfz;#e5@(d#m^Z(qEJ+`*hSO4gZ(2oCv2jF zj~lM+y$E+`-K{-R^Ytey$EWH3*l32fqhAyRLI*=23~1Nj*sp~7LgUjeUVF5s5ZPsZ z3?(Sxn)CTUvk&EL;|rLWOWtY)FwH^`zFyj8gP?<*G7WDZ;dr}~>42>)Vi<#2Sr}i} zNC_-T123($uVyKx+L=oZFc?QA)X#lyS;b*E5Pp7eq_*hzjsfNvJ)JwN>SUQzJW(DP zWkk?L@v9MNa{X!4f4Qys;y5{jCkP~r8%m*M&onUU!?>kEaisrngfIJ)I!#HW;ODHQ z4?^N@MDOirRmT@6LTGQYF|ktKflio+chwWxnOZ=2%x8arXJ?!R`a`($_b2QQoh1rH z`hC%-8Q~yF&XP%)Hwd4Z&LKSx$3&2)WarU(X}07DF?$lBhiH^&Z%_;AKwu~OO)K2` z``jI=8uQp(a7jgq+f|$EbmY|=&>wxlWNR&1`D?+&?$|Cb2~X`;=deEX7wdq&=5n9& zJ&jjaF4?arw>FO&A1h9x1ZCDDQt9-xNYfWMVfIz4eMcDAm3;Ggj0RbCxUo*io}#TH z{$|!Om*&(|F_dnZ2yb)sXnSD5A5c5a7>YWofi&!=;m3E4?UhN_D^MsQn-dw`hAP5Gpc*$Yg z^^jFoxI+34B<1ZNRa{a@^_w`1%N{c;(?U>xe#31G@0OmvP2?z67tKRq%5D7B;exSU zzVy2H^YpG*B60_f;d?@+vsbMKY8Kg~W`2g#>Icl zfvEcopHWx1H(e#9Sd(lE=ajLCFGX|~shqxxwUN>U6Ohdprv6H2dB{|7AxTdMP0nKP zXw(LlXy=I~#a?nsuwtnrlaogKVZ*Onm7$_PFCm9@4{l;IB1;4-B5+vOYCEhmX&qBw zh*&0t>pNa~m$QahSd;x>DmOA}Q~Q1=4zq6-hYP#a5dk7$7LoGMWDjysM4F234?OSn zvy{Tp7bz(5I0Kw_x14u9^&ai?QXQ=Zivw#IM%E%YqQ7{L_qiQ~qPXnptrS$HQqr96 zT;afn5T|8V>~QTEOtq=EHmWiS#x9hHnfZ*fpw*5w4vqq-(bN%3>4F(6`9+!PTB+#iOh7C4Vl8wD>-$i-4~1 zNh})WDf5g=$;6g-2CVglZ|Wq-zMAg8BTf!PWDk!R8Fr@1m!4`YH9evEOgg-5su-Uv zb?nYXBVyiHRmn2S04K%RR(u>gB1k>MT^^5YP%qy3@hw;B_iQ7Fn@u+0n2^+E$5G+y zff@R%O6+W5LB7)@toM1S99h`7nJ!9&F-9taiIk1ok(Waf%8GZGm9tek3nh_3DE;s= zAL-)OYPLRXo{Y**eq8)6(Sqq+arHG;iyUE$i656K@S7K^tJS2qqEWrLM-;lYX#3ZQ zqdBQ!BWy>X-Qu!+&Gf*d4Pz+H265(;W@tIZ4*^>{nL98ajnzSW+dWZUnoYS1TClAJ zvrM%^eHr#eRZ{uNvQSJw1e5gDX}Uhirh=2OG#yFwXws=dRaET`Xe*egtf6<>P zI^t$eqpBga6SD}WGk0&0m@Srb8ljl;0tu7BMjUNht=1&IXF#ocCb`%=GTW(Q$EWg= zux`OeQuE(?Eiy!bzSpiETZG$@p7cL$nq!HyMvAeqXtodzU znC-D=T?-z4P**df@KmF2#h^A(VOs4~4y4bvXL&|e!#Ej9Rs0%=7Z_lwWDM{qlxw~xa zF29tCI1>9rptY;LhQ3y%o5Ki4G#(jgJ7C|E>CfqjXi;9m$D@vXBW@j-SiFd8B*!|l zS*zra+2nM1ClTB`lR&CqjGFcRvwRzXt>fk!J)TfDFGF{gHDT_Ry+Lvp!D!on)stgUh)*{7YeuS^{8Ib!1D!?+itrYDfRNBfC`O)hekp+70jt!{F_lzZc)KA?7)c9+@>YZaXl9x6u{%2X zs&VK=(+Yc9^}rqis#CoAI+P7xTh_+7^V>Apxq=I=R@dokV|4i30_zHJ{ox*?>5pZ5 zYT&7~NfLJYCEq)Yg<8M4>5Unt)NO4ZNwl&?NCvINk(D>!8J$Q&r;$sXue;>+ zE3_`mb&E9n`47$TNAjh4Iq^<4m0TCRbJmC0{AHFuc*-5tZ&a({R1H0tl54np8O8Op zQFp&K@_nV742p)#m*}sQ*sXy~@AHgZdp@dMl#aVsOAHe9thg4l!(xSJvhx>`nVPc+ zAaeUtI8Mt@U7Y7nBp@bcPe^82G&pDMg_%JueIUzl;j|1ezg?f!cOJlW8l|brt58jv zPBPPB2}y>No;oy;wCT^1oKTjzRi16G2oj)o1QPs+*Vk|171U@VHETo4wCk=Qnvej7 z7I0%xN3(#4?d4U5b@I#?v(nr6RUbe)*lH=PI4VIgNK(}sXk+Pn%o!>&8AcTfn@im0 z@Wy>9SHOUP-$sQ_iB_Cdq$Q>g)o1S)Hw*rDj%-QcSy~ohF=61vJ{*;B-Rilr;)p26 zsw~xaUUJL#qGBal&`XLY?p4(BQWuv{Hf|E6042#|Y!V%_2Szcy8aQU;p+=51Lh#|vrO*n`uN;Qp zncNQHPzbHz;1O++2#GJ+u}OoOK`7AlYMzMQ^;~i{t*8VZ;r&ZWusnS9CCIJ3ye%^3 zdK6lARWtB<+$tOs=hAzP`f7b6p%(iwdvE!)Z#hX$=?Wy1cvvxB(Z1aKbPZ4pLs6hPJH$XhwA0F81>N$J68^o7L_7y6$O>^BLktXt9`IZ`YoOB44z;&3vlc^ z>vb<+*&5jfOKTGku1EN!a0!AF&k?ymaAdr7UGvf`ZwY9RAx#_QlNJbMChYlEL8 z-X}Gp7Ew6}i*}dE&5UFW{z}M1J5|=hU3X%qo+lqXwCMT~IJ&sH*map3Q-mBd>HC`!Et*Hi-e73(gdti3rq3!0qv-(Us}}Z zer*l6U-K%0zr99t z=_Xn-@~EIG&bdyQl{yvYbMl*6do4slS&ol8Ft_x{@AVOXVbG}NM_*7*w2 z`l7RKr?XB5Eg>sz@1!4{Bmu}A!|lcVk>z{H_1LDYM@#cG1kQJ?pZs9@soU!TWi*MP zIxBJ1X{KUyfuEwbE( zkqO$p=0;ue7jk^TRW5{$6(Z4zWEo=GA9W&^a`Ai43!r@3Au(8wlHHuq46!OFRI z(T_uDAIQW`7?&UWUFXC57U%4W=(q+;HHW$iM%y;Absbr>Mfnz@9|;Kn?s=ly2>7?3 zzS_B0ZOof*uvMSn_Od2?CO$|IHMq(a)%BYXAFcTaKocDgrgDN2GHkr*1Qgb z#_`32kI{tW9M@R9>e=q50c_JJg8Ex1_F|uCC9|!KjJpgLV`cTA)=FH^8kpXLSz(*W zVggDoZ&$AP)&ZmbvLGyQg4Yvm0z9!ipeh>?Z$`cVBhb(-Pfl?4!meRG<4Jsb~l(H38zSsbg1&oRVQ3s$BSQe(_^GIusf6n~`9)SoG~suM6HO zMMSXE>qy1HzoUs}o%;wsYR@SL6U*w#GyW}md-b?1W;hA~1_VWfSKS4v0M8`AYu7N) zzn@*lFaHp(CUGLA4^V-PXOsD(7$(A5D79sN>TcO`(la8){-e_P?KHjTX!qwh>LTy@ zO$LtP(-^We;(VNgK0Eb|?P|u3Osi{SZ=c{V0Z8#%>0C~tL1~eJl%>Osz>cjP=B&ub zFBN7v8hdPcRTb>9Yy#me8h!~=6O12hvXcfw1YASA;v22ffqFyPyw`f|Z&TI>%*0q6 ze3Q!OR6AKTi?G@HCDQGbF%`m=!n|&O2@JNVn*O{opPo_3(E|ZSP=JF$wIum!f-IPf2 z*7Zc<#C{4nsJ?lnb}s z2NAb&`MkNbdNM1<<-+1OwQZ>FWSVr!(PGWY{L^YVu0l#BTFX6BH%51Lpe*&EW8g{{ zyK)^?Fh`=pPP!0BmcQWq(;_B!e|jAAg<|2@P2!=8z8SnW?WU-UKIeP?I&`Of?8B+W z%BX)v1`-$2sp$nB!a5pPi&9DdNb&-uF4p~yIPSqIdtj!HGBMnSkXt+XE~nJe5?=@g-#|Ll?ciKX8FcM z>y)hRS&USsujrY-;P!}ukhTjmYR}BX@dqcpE1oQ+C72c|)1Pta{W{)T9g0a4XHi8) z4Z9Y~m!P(@+}|a@VbGV4klM)vx~;d38a9mP&y`kes@xi56XIA(g^$7G7n&3xPcrA= zp~_q7|8`pl)PdO)k7hEORTMU?(8XR#N)0c9ik>^Ng0+0612)yJ3=y`so_*=e;thjx zbVWUe-39e>>tLx%C=w^-EgIxF@#!V;v`m z^^BWg@mF7?aa3x|5Z+;#^L_D3usHe2YDK7gLKroeIfIXGaalD;GWV4MH^4|ewwR?? z2?|c)hyLB+4AJNM*9%CQL#P864Cs2vjb|W!{hTotto=S{{Mhw63t3p&*ON1)R^ea5 ze0A4oIjvJ`m{e+Acx5t~Y&=4~MR!Ct>)3~(Rtv&*qZyQLy=F(#AbuTFkP_Rn6hN7TZsAT>jAUacpU@a%5&5XXO@!(LhYEpZ%aBVdA#w zD45?$Wb>OMp_Y(pZ#BR5t|k&fV&yKpiG?{8#@XKckEa~_w9>-xJJ{r=W0TeTe$hwb z{)r=iLGS#TDZDDAe;4sgQ zqH)w)8@BVL0ec<@ysPhM^~ED%FdAL)DS?vJi#lJg|=y8r9!3dg=51~1=Psc znT{=4;e~N^F0>t6>i%F?H}$liBbt?%sA;F zr{aoWG`@8j-&}3)s;;<|OAOm^XlB55az=(lh=T`t@XrIhkxyjJ#~ z*N3ku^+%~?UR|42urqhNT36WMK|E$%t=G&x-=ON!QEc9dQ*D0%66Ptn3OKY9w1ja?h~Io?`^JKi`07>xk0T)JKhpt-(S`8TumB zZ@i6Ar3&{n1vbMDE99%jF?vVPPME!tlq0=ZWb~*=l*1K`%NSolFyjv*QC)P{cNTiY zL8Kkab^;d0X}r1@7Q4lm10>>2XW@YXh=M$oL7OtFJ+Th$>&l$$;1I#DKnNmj;xpI7 zb*(eWUgKpH2ASics?dV*>(E&{AnHRQD%m5SB&i3Jwdl`Lf0;}W`eYSZGcW4>vj7zF zt)C+f76WytOJ;?XkkgsUF|!XvBAt?c=+3HjQ{{2A#Q9!unGXAyIz`x8Iy{vdBYmBn zC$FHZ?3zL)E8%**hU@ZJm}#hs#KCt>8kfij<~ag^&m00~xi{%eI1M)OAqH|V)>z-a z)t7Ik5}{alU3ZZpleSu?rBrncqf3jpXZIy$>_MtvcynUG)DQ`mx6OQMOBOloq`R2% zam{qS+hF?^y|$VXVss=~!WIN(nG#tT;yu=#$`Coz42*dXe6wc|2UNqM7ozaiZG+B> zH=NpYVP*&~641&9qHoB{C^Y!JMX!51vo#aRp;$0A%!Hee?+_FBQ*6e!iXfSf zNY%wI&G=UXgt3|wrf=e~{9D=Mo3Kc{*h(x22-a)hw(TcZdx=dqDh~38GxIjIy^pzJ1heJ_;T&hy z4O-Vs=bx|n1Ma`NA18oj{`XjcoTG*A{eXbEnGL_LwYHU^g^l?i0|WQVlA5OXkBEqf zON#JP@JYzZ0`JEQ?k5TKtpWEV2Y;-=%WLicP@!Z50w`%2XaFoMEC4137F7T>kF~b0 znGJxE<$gszO$%XNLw$pPE&lOS{&HntlKI{^Qw@x!+GiD{GrS4i!Lu4?tY=UkeOO0BQw8 z9UB8{z#n`3zYYKh{P%wRx|8M*G*!;@BCtT?M_b3aU zGz~~|stt{9GU9TdE~K8ju&OnTEC&jmn9uS@sw_k*S0+ALa4`-sIzl?15PEPF4lx)$ z@R+a=R38I4u-*woYAP`B5X0S%1^6F9tm;r;9{$n*|Cq1&*D%gwNAdr}P5>;VrR?vxkx>vI!TrRjqLnLwhVmy9rZf0`6H;G7>oBW z3O%3IyuYKqK(=@U^%G<9{*;?#CPcJS+c`e{N!Uj>VSF2ew%8n8GkT zF%}u0W3i?4fh~4CrZD#-QBOLn8J}aZrSpL;PCTYC_p?Y(L;W0!Eu9Z+apN(CVSHjN zGCs#*OXmYyynIYy7@ru6jL)&y()qhBvcWv2F!x1SPr86GJ;!28=L1{BMz(rri%d_9 zMW*LiY?*jqi;<5h4AT>1k?A=WTP7aZV&-EC!}P>hWO|OpmWcSZtYiV2hiNDGc)yW0Cne z7F#AB*y7b=3d8)wSY&>V#g>WRZIKQ3F@<4%Vk|O0$70LG16#y-OktRx7>g{=vDh;4 zz!s^HtsmMV%M)Xfg{=vDni3z!s+-Q<(d={{MLa_)lKO1>V00|Icsm-)iQq4|H+sF^2)t zJTVx7G|$P{()vIcuOG7*Ak7n_5lHi#jV-OeD1G{F0Mr$29#|D*eC;B!E>j6BfC>BpP~2z+Wl z0-po2W#oZCZa*qDz^4Y}-@6j$FS=$wFvy!nrRJ%9IKaPmApBj*V z?;@PP2=)VmltH$A#2}yAf&+Z6!BKAMd0>#;k4g>QQv>q32VLvcP-bbZ|{;2`^T#IuL`+-5qJ}Ncz zPYuZDVw`)}4-8TV+3pd8d}=ce@VOdCxn<^oLC!uZHBarv0X~=G+{1ogkh_mc%~RWP zfY0?f_pl!rJ?sYtY5%CyJheXu_*|fK5Bq^ZenED4#2}yApaXoa&{1v~`rRPeKp&Nwr*`N7 zpG$P^VLvcPCiqm>DW*7BUe#yPib=)-g2` zR4_QI&u2oU!#hxE0iGs|JSn(JaEQ?_TXmP)bXVG1L7;%ZDmTzjQb%sM zZ8%?$%T#xGWKecJolX&?Y95)z7QHXLa?-_(U5jhD#??TyL1 z6G1^5OpT};kiJ^ix9>gGkv=y$bb4`fa&o3>*q4zaujCN{V9y(YG>tF)KBv~EATEqI zyMs>PTSX zPHrI0jq3VfU;O>UF75=yUuz*vM3Vg{46U?SDXP9gC-XoCdPc)5&@ zxcymB>!@i?;|JCRD9kwr3&wyntd9BObgoK!-F2-HU&baimw5@^d}?V3%y?(}A-9(nhciP-e|&_g75`$rE^CAoLSh z&>GTe1Ig}pu-C%`Nazp%3nADA5Zlj$!ndk8Tc25Pii@LzJ+EGjX({f01=DKabmN+B z8G`ZT|8a4Hb$j)R=aT$G^`A@alscMkw_4pw@U4h zuy@9=#cpdxgz+KnGf9TU9mj=V>+T3Z?$m!Fg4`-~gM8JcL|wRJ`f1vl-iBSQ{_9RZ z=T7qC4tT8)c^9~R#}V>5`rGs#&E4Ml9m=bkmf7pu7aAg4K26ZvcwZwye>J2;4REE7 z`tmNQ?cU*xw!0VeN((aA+_(lf_iZU`l5HZEt8ccy8-UmFl z6?ed|Y3R^CXiMe=8XecsH!2s>+0-MU%Ppuw@9~>6s8UaU2*yqd>6peCSXSL=hRLg! z(SNqT6af+66OCH^k>$sc?x#sQYmm9!RP&u%uc@$uu%1zsB?WJ|>=4Mj=}JNFOaN{! zS#Qt6Hwh4ayY_za*q1`CCOANeY&#z+?;?yctB*jphL|a5JSnjb_0^1rzHbD! zVNZ<1O6_UCL-7afneG@K3aqY|MiI83qhl*B)>+ivu`dc|Fa#a6nzymyTU%&q0@iHR ztCuTzXA{?G0c7~BFT6Z-b-`{$GV(gkzU5?)gcz<-T}nzwTbJeB!qN%DO!0;8k{UU7A6e?F?P zeu_$04kP8;3ok|XL@sqgahQ)iPPXRP8^>Yc8=vr1rx53T&)W2Bx$-2id!b<`j8!gKMjT>{sQpys|C)9)(7}iyP}^C&|fH9#gPFbf7XfYE~ahXHoA{$yE~?H(2I?gNOXlQ(_+_+?pS=H1jXba9OI)yOH_ zxHFA1X(5z;W1M;pC$rAJ*7d7%>EaN(&AN|v{T%*avwPi5~tWNjID*RFJS95q_Q1kn#B-V;o@A}r$9F- zRl@(VIT&KSPuqLLe)hbv z3JiM(*?%gS)@b;Q-fK=)LjL>S86W$kr~HHt!r!7O0o_Ro2v=_xz|Tpc6}E?o@2eoE2Fi2lTgao z=^pg``;^r54cq<6ma{zl7r4<l$*X#VosT_w%s28+v*i6X>)g z@*O^O2vjjvuw6qh0!m3|2gg#Pdg@ znV$Va-~&xP?pM6C0l`+>9ygj{HZ=6IL`w@ile3{wK{O#o#S=!^MDOKXDcN#BD3P6E zJ;-hrzhn>S_XuM3!X(Q=qR_gX2xn{6oL8>NN33u^m8`z~v>#bY4L2#;>J{j(RKe*L zFDpw_7VpQ}8mD?|w(TYLy=-;IY!bpQy()xxaZ{&mqw>XI$D+6p{Aj)_duh?2NUR1M z*^Wx3wC`YH1Dh9_YsJBcxSVxDg;Ych_@_z}e6fxLPXeRN*hdO)`#` zp30YQdiRjUYT){6@V^k-ASAC7)({&N#0UUf}mmy3(xX`Bu zJ%T#)0Rnq`OLmpon;&7wS2=vG?TrE~wvT>Qiqzl{qC!K}V@$Z!&E%>VdN4tU7D?lu z_n}*3BYY5S7x&@2HaVG(6|OtilG}0hA>`^_?m2^D#+tI;)@hSGD>I~cHA9`r2V}e< zEUKErkC4&pKRG&gxn`IH1-vfrEhy5mbvZJFlOwH+_s1%MadnrJQ%q0@V+cdZYA{sZ z$ELm#f;F#>BM3I2g+k!+Rg;~6$0_3M1R6V}Xs~;XHDMQ{s{T4guAwY_o#>VR7-cJ5 z;U{xIV=cLHg|^A0O#%eJ{&24cbx~BrM)8rxHL-Z1@F$v~1RdDDA^Y{%VFiO);X)l( z)#LMl3dr3JlP^BMEU@o74V0`HcF}giyj1WUd-dZs2jYX>b%2Y(bxTk$2imnjQ@b~T zSoLZ7z2uvmuTP^-(n)UAu);*(cERmR;q9`K<2wp6$kmB#1}D+bCCM!k72aIyc2b_4 z&#l&IR~~ix7tD>8yqlJfLRrQeLD66oTXQDT)r9jTthgejMrbbBMr*^y>u`nLxfPZC zK7k$0xfHTdhLX_T6%f8k5MQq}jAbN+&M|0>zn|&gr>Jb(D@zMFU1V#qkfutR~?>#7*u`M{xRXm+i6Ey<4Jxbe`6cBz$z^|%L5Wc-J~O0=@@^c?L7?T(Vy zj>|+^uGZy&3D4o+4&H(Yqe`x0F6VxKyy$8C3v=fqfH?^O%O$J0dC7(Irx(u@C;2TC z@P<5)jm2CCd;D9>yf*9l7lBES% zv?eqYnyX@3?gj{Fwg<)PN1Dr8>z0F8Bq8P;P?QJm$0IYHg8aB78(l0={vHwyUzsHf z8?Ek4vO6;XDX%C$4M`OmC@|$kKMv1`5+9EqInGuiVUX)-)=5T|Q0I|pDjL7z-R64O zd0Ej0UKSqfTXH4iAW5y1{n51N-gLmJ?+k2#(XNj1l{N@SA`pf?nNSjAqrExpB1l#K({HDraH>?C<#1Kn+pN*5AYLE~Qi?e;j ztHK<<)a`@B&RHC-U-@_dCK4ox@4Oi|uXZvKN4; zjL%f`XISK(ZXXS>6j%Y(IrA-6*&1964sg^}Us(>NZ52ieRC%R#-6EOWG~hWc&kk%Q zTdQvRU3xV89?(Ry`nT?cR|!HKEs6yPy#EzZnt&&eW9yjVeeju@*m#hD+T<&j@zN&+ zBa*R^-tjFd@7Tj4E1@VCNoVvL1JQ3{>|?oUI>uXqdd<8VAMW+dsNH4<6Kj?HZlz5L zXZ?LH%@%oOTx6xpz&IaunzZrPi)>b?O%NPCuqBxHLq|J>Ags=O*cEEFmbq_yhNpY% z=tIn!CttIwG^7HK``$XQ8+|C*GWPSE<`_p#pE-UlEFWDGJKP+wHv<8(`FQIR6nTM;z>w z3|2@1y?XQI{NVdxCIym0#z8gP90#uEk!Bv=`{AnNo!jehNut?WjIoe1u^Rp!#n8yi z0R11YvJlFa$Za=NsSZUf#-a4K&G{-P1SbH1<-bQU~>PNn*!gU4=Ru>_zy2e5n#t;vH_ z3SylZ^0FO`vHrn4I9hfl=6hMiAG3T!$4Um^NJu!%yaq9is0~zAP@Vq*1(aLPKvDO( zA|HuOtTAswU#Hpg-$3XyO$$ih>K55MJS%WE@xB!CN!ngtxCtlV_xYlg;}iVD-Kv9T zzChl)dQTU_s7#{U#@1Efn`+eA1mOQl<) zCS6f_^A>zDUK2B;wX&%uuY^V(*4d?K@-o~?6o1I9%>xN7Q1R^w=Fbmr3&%8mZM<%O z6J8lwxFp#%ALGJX1&JxnX+0OqeX+c?AQ~0~Je6iLP?K~kWpNLSQLb_7qg6L#@^2#7Nw_hLWsVB|PYa$SXO+MiD*vg1I=dDFV zUF$d@bLUM3o3G#i%1*{y@h~e8JIq!^FJ2*|Xq5PpUMoS>-^6R1zvXAp_~kfxl|c7x zKuDZN2%2n>8t+Sw>QH-~>S)iDxJxqnr6;u;eApR1n#~+NoiyQ$$(4tK?}i2dw`xeL zYH?c|GH74l?gL(wUbdn1SV;ve0ml-hG)${+6LZuL;+rq0<%wf?VQZ??pt98d2m(k- zGJ9NO*96~nyWMmXXsI+ob*c1#0oBLuiVgt@Gr9Z`(V;3;M$?=(hLgm{6ve{8teIj4oAKn2eE_9p_ee3eCb3Np?-G7{?b{vKjoT;8RPc zqlDmx4f@9PwQ$=@r1qsUS_LbdI+mZXOyAP%x-1QMjWEl^U67;&foWS5S9Bjnt14sJ z;+G$@nf^>_EJr2_APFhL;OK^8=WiUzGB&4K`2^>y2E@f*aF}~DGb+@gKY7@0|86y6 zE&FXEU-w4gH-%TC%YK50O>#BXcr?O(7;(&UQ7W8zrLet<1_lPZ2C+Jc4t!~~Z4Ri4 z@YHIhBC3h{FgM)<2I2!44*8wv?x0 z7xUUxo7&REk-Mh46}Gn;6D zv0-kjwO;^M+;5PKioc`g=*L0@r*_@Fl{Gd}wOE*Mza&EbY$Ufm&2ZgkWE#MTus4<* zdhsT0+jV|u7T?OD(Uv-Ix(HSx%}{>{aa8O_tVZ91mBH}iDU!~ci9_YR9<*}H`af(S?y z6cdAhNRVLyIZ4ivRFEWyWC3B297IHN5=1}=f+UqBf*?T@1rs?)M#(uUS-3Tz3`gdD z&OMv`J>R>}BNzF!xA0}z7Y5Duvre=3Sv)9Iz)EEKFW)PB8^d#l-~1JZ(Hm0T=l*p ziW<+pQ+X}xcBr?qvr8cR4)usYoTUa7}%+UYh`FCO&b{>nj{a$E*R z9oSL%Na~JevVg%94SUtha2LGXw|fL;ZOPx?r71P}5=J=PrQxpIo0j~ld)~}Dqme!- z_|t8cn(w{_x!HH68e`?JxmeD3J$Zv){rN-=;oB-iacaYX`*Z?QtE3H&R#`8}GJbK< zt4X-*Lf3WOZQ^wI{Q>xRWKUMr!ea_+uCq(}^IFsn}(JPn)h7^6-b*Z+mP-d5 zqCR{6EQ0GkG-wa-{_w(ykwi7`QvCzBkPJ~O_2qj#MaDfZR>)=MPu$r){S&@DWC%KB zfTR)1(AMmloi+L!DA+%1U}*$_Dv4iIkdkGWP*PXtTmLp{f`XtwmVyl8;?|B3?bBdY z^JzG600J>~;1FIOm@Wh=hB7p=vV-t|nTt>fgKM%zW+tY;cG16rPCJdB)(DleGq`GI zC}w4H)d>APR^86X@&W|RyM}81DuV$9${j z+xk%+p~8jpCee<|D1oA{+K9;bhO91%1R;zD{dlp0oacE{Jm$5x4SS{bST*AM{`Y(mm_^-lxb7ImjtZ8MaaIj&G z)$#2=!ul^o)!+t&@x^!5g&RQ$Sgd4Qi}@Z~qC@e=VlQe-*7a=Scsi zH9qAE7OZy~Q3L+Zj;O(h__wrH7p6uiVA^YS0tI0O1S0~p*96qU3M3K)QFlTQc$@CW zuWF%ya^e$&noa0rGU(vgmQa&m0~~Ar$PHdk{x7@7e|A)jzY%bWKLM|%CdCFg)=u(2 z0sJrZkuWY!e&Alj%L(4VlLy~zLfm4qig() zfE)e^_!Ko6Ho&p=m;V9a|Lh1Ge}VT*BjBh%0Y_3(VFP@po#sC~;>OU&pgw>F@SR59fbT}$03(BcF6tMsp)$n=_D(y@-N+kT zhhZZ&#;XKwQkh``d#9b{ZsZLB_U3@qZ?MCu%&~#J(~fdC@&*9=&qe(LkEkrLfxXjC zayRk@0Q=8H{Q`N|0@A~G8hHc08+ikO{pX^7feLJ~8{s>RyaC^hyaB-8{C4)c&HjQd z%n5v_kvHJGkv9O?e=h157{cZV4Bu(w4ft;44FL9^i~0o?v6*MVcN%#Ez8iT1fW7%$ z_*XL`NQ})q3%=9H8}Qx88(QV>w<$<3tfFS8kvHJGkv9O?n-h6{caRZSMa@nlZ@_mW zZve2jDad53qGqR&H{iRGHvrh%6l6YDQM1#?8}Qx88vyKW3bGQbsM%@c4ft;44FL8w z1=)gC)a*3!27EX21^|1Tf*i&wYIYiV1HK!11Ax6vK`vnxH9L*G0pE?h0l@xKK?;&! z6*W7JyaC^hyaB-8rXY`Ea|DL(H1Y;~H}Zy7dCN8hc^9jw*=ghr_-^D40QNQonSxc+ z>@@NQd^hq20DGH)e1TQe>@@NQd^hq20DGH)e2-Ps>@@NQd^hq20DGH)Y{e>Sb{csD zz8iT1fW1vYj$jovJB_>n-;KNhz}}`Hm$8bPokre(??&DLVE?Hg1xc}rnw>`8fbT}$ z0AO!ZkjJo!nw>`8K}Qk|F{`-M%2-N+jN>}?9N0Gp^m>@@NQVmI;z0DGH){D4i=Aa)vg1F;)<1Ax6vLAGHN zHHfW|H`eV>(2gIl|DYM!Zb+l4eA(Q{aI3`%+HnLptZaDBz<_@YSke8zV+AlT{|?r; zCkQDDtb3BY-KHQ%v56YQrm4-p*sA;|U~e}}{1ZXifvHEMx$UoXh;BRV1wkgQt*hCEi7%}Y-?7!r;^Iz6xZ&Q#^ ztfB_C!(|?bzvj2|Uk3Iz1$hQr>_)^+9d{7_Y{#9yrx|J7rXUNkiW=Ysx3gyCpY68u zH-WuPL4L$4YT!HVAmO_mWO?H@1=)^O)WCPzLH@H{cK#-2Z&Q$CSVawdryb-!+hON# z0(+Z+{E1c6?9^Qc@y~YG`J2H0Q$Y%nVHGuqopzA_Y-gRn3G8hOk{+w5LF}}H{AauB z{7qnQQ;=L(MGa!79ppdTQRi<0dz*q3!zyYJJMAF<*={<26WH4nq#Cw>^oX50=^%EU zbhOGpZ&Q$;u!V{|DadiGq6WFs4szE?2Y|gz zL9SvIH6Vb(&IY1(opb=$e=0~pa;&0er%pPET_+s?_BI6x!76HY>ZF6%bi7 zAaq*mcqYU~f~9Mp#A7PMvfRyG}X)>}?9t zp2`7>g50T-4r13yN2~nJHU&9}Rn+X%Ne8j(qyxbIQ$Y&iU==kxb<#oXI_Utg|5T8I zlvqX0PMvfRyG}X)>}?A21XfYAQzsq7u9FS`dz*qpU==kxb<#oXI_Utgw<$V{|DM$ybqGqQ~I>=oo z9RT(=1?fxWxLrXacj}~r+;!5?DzE=jK?>qx6E(=4I_V&Hopb=$+Y}@fHc^Azsgn+J z*GUI}y-h(fViPsUojU0tcb#+q*xM8&5}T+&?$k*Kx$C3@z}}`HrLc(_$fRLM{J@7xl<<{By877 zr@Vffg7m{GYOp)$@Iv70ZLyk&2n4EPYi+1*WCziPsz^#hpy!Mn?R3F$BdaUG>5>@v z*Q|8-Igwy46%QxyJJE$e(Oz`xZeB15RLQ{3*31#24Fl^>xFG9)K@K$|l=XF6LnG9` zT5E2p=>JqtsI=KtJ0n~0BUcUVj3kW=t*;nCpo&ITCU&L}UcP^8HHQ#Jp}hNaI8!M& z-FPKX1Snwz6uyLaC^7+gj%VFzu6FY^J9H!LI?W8_G8O@e-AHH4CH#A=|569}UxoE% zi~S30S{W)FY*=Gm1^FLg{g<5PxB;V){|k7{`#11K*^9XuT;Of72SDas?aKTb4j>G-1HDlLtu3Ukw@@v}nNl<<}ni8Tj4O@$rGluh`nc ztwagRBRF@i`ydaguiZVQmtA#jXQ%AF_c-{Zu*TYB^$#O%+_00ocT4T&t@_R&$mve# z(&*ZJ#*fm`OzwFR7g3k^k{YKuIzlZaOEOjk8I*4^BWd;Pcl)Ra-=~>bu-n++pMe}d zOW}LO%TsJG@e7}o@iP5duYv;cV+L>565kh`DVy4RpX1PK**VlL+=$LGTpb}CPIm)N zTmuTCvi*4beoEQdQ4+qt+4(RMhx_b}XN;sh{;wguLw)^tdHJ5A9N)jN+-SW?P{z!} zRJJO36w;4lL5%Yu<vfvr z=&=W7T+maW59<@;`QRY?&hvkZyJnAb$apb;DBt0_tMA?-uZMH`vwPo6w!Ox2N;koo zk_hnN88ZKdcl~CO?Mv}-5AJ;{cLxmO`}HH?!DM6H~8=&akM;T zmlTE^`NiA?J?_)LoSL<ryZEB_6;UrkE^{Lu z8JCZygu$V;D;``DPgl#xsjhiC;{}S56ZhfWa2NLQ5cZ&mPG-JQ70r?7Vwbs5OaDXU zYU3sDqBU0h0a2e@1XJBQBf;b&-Z)Wfltue|$*)XLmH$}dSnOt4^N?S=+rP#+w{}{t ztEy|2*K<+1V$BD~{zjY2PdTU1tRpK12O5vk6XMOiI_LJD9{(v}&aIdHD_I)*$lMR@ zV~zUZ^^|{AfGF*WP$Tu_^nKpiEh`UWM8{sfI$3?3*vnMlrMQPKoSXqtyS8w=;`5t{ zZzpWrLI)7>xJu?;&?cag?q!~Ohh;_YhJf-&IG)w^SX9}Hh7NQd(~+1JhXP^ zQYZ<|tre0P+#9tXuYK;0trF1Jx31wA9#{LxK3krKb8L;E5cf=~hnOBtZU3cEdYoHp zBq9;x)8u{bvi(1fteW(4bm-_Dp9t&5^Jr;$H8rU~f;W9jQ~$G~fR^vAw&q*z#c6jE zr@povI^gCS>TFO}1&gxwC!CBt;+AtKAvPNJM2$_f;T^NJd3y@yxNBG*-}SLvJq7i_ zcPINUo3^Q3ViU+&De&Grc^;`J`0`fr^%p$&nNanr z!C^^q>BP%Uxc&F&o3z`X_#@8Ao$~%}uaWmvX`n>lh%3t@#mNiDdL)`JE1u|1OBFp& zX<71Cb}H?y_QddHY~^I)h%cF%J$V=3BXY0aPk!IkKF%aw7^xcU=_qEm?UrMif@we^ z80s#3GH{(fsbma4@$_JcV{g)sv_VwnqQ7d0zJF27P2t=JM?>V61+TC>N7QQA~8|9^v@>`tYpLCM%{iYfF?8*@rmUB!~%{f9V<98%&sVNlQoXV?W)OA?v z>#scFyg!k||3S?vj$4ZrOc7-2Q8x=RC7n)-A0MfGz51E+K){^WjUW5E8R5Nt$+E=4o-$Yr`dKP34~XRqdE!#+EnJ2b){(me`h zOWqC)GZ73;KeUN58jEQ&_|B33YLst9tG>T&WTn-VQlr0t|gq%tV=wZQpYnl9A!Qtt4(2)*CxPQf6x?I`dfI zXyiFc1XO%n)R5-{Bd6=xFSpgd3Wzf--<32lY}NcJbkXmkc$3shdPJ+A2m{v`p8D5p z%D2^m3(d#IO*M%#sOOkY=pR zsm`4n;yDHXbm>`^l_n zUF;UWtGL3bGw?2rKkG1Yssms9>iL{V>&CDz_~w3He%C*arF0~TBn40VR-TtnO#Zqu z6Y)|mqn1y)p=W=g3}5)D1Fc!zIk_T#C&y~8qKaivq6_^$Gc z8a?E`?}`p_Q`H2D!HO+J9GPxnMr0lym1RaMJuzu&-#9yL-zI)TO!jA27IkYpdT7Mq zH0ipQDBzF(DM4gjGELF)wd<*K{l6X}uhlRep=#p2jwdE+FaqQ664Taow%3^7Pl_2v6;vX;i8#s>{magkaakMDhO zq>;Ih{ZN}XK#|ZGC%^I;bJ@$!;d%%0Y-olHI&3pazLCWDrBj@8Q?w9k>Y#@9wXk0x zQh8_CBbvm1O)=yfQyqQdp(9%A$#eGbC47tV_*jQPf&pqnOhseh$4?c()A{dA)LAxoPB zr#_7c91SgONi`}{R{Z>UGQJX-NM5PC#6T88sG;>xme@7M+rLa3{tup651EuYT8er>ZQg8hh_A7BMSZrTuAxf?`mjV~@z@IHhrsJWfJD*c$=px4GP zQBHX|B#P#~ev6;LDTuC2ONL7A;xR4#2%dntJEFl(<(K=X3Oh41%)~~0OrJi5H^c6( z@O-~l5mBX7D$n%2rlVgPT8OGhXLRW-kh)A3GygeGe^571@z7W3nxcS9WKm{)!U&K@qo}oG`?^iX$y#1LCTBi(rk;}GX;5)i($(TUL!}j~bIg{d^@R7{ z?3GzGp-|5XVDLYbAK2KIJk@li!)oEDKu|XE=y2qD5^;yvtC{N_5=1lj)~%ubDkbTODAtn!Gcq zA@0hM?Mc#bHmSWMhIsarqS9|!oF80a6dER~W_wWj{e>w0$hQmMvhKJ)Y2yx!_S9q$ zkZaki8#j3aC5bF*vN?Jnzf!9DA(R>@5DGO{@I*dmrn6EmZyb&OiCtN+lSKYqbD5U=}1 zt%L5O@)11<{b_!m$nQp>m#>p7y)|;zU>f;eVlzMzvN&x1y0`|pzc78urM=Jg4r9#( zo<)A=tWYmseVfbO6*^kir}P(mrsMp}_K&2#-1Fgetp<|#r>VSfpv#N!+dOrvJ;R*Z zadyu=Ia#kCYKIi(HNW#AC|}(H~hj`%d{ok5`yuMgMq~wNRi$)ghC_ zDv<=riA2_<@7_^wDmV|F&YaYgGQ2xFBV_HMb3?}Hf;qg%S%95GBob0db-{gr|C2^l zL_Nvg*HZ{y`Gz)Wq|FcL5F1rcJ!SCW=wV#kBJ~dS%N+M7?fBXHC!#`MG-W*y&UAdp zy5cN_Dnw~FWD7(DRXmWQdz7xurk9!%Nq2F7IHeC#i0lqYTxyDVl0=cRlA5TW<=IDZ zkFxo*Uj*dg8-^_8IsRn6zQjcdIlyku$f~~gv`iff)yXIL_z*#>XN$rmqh)D8{=N$=J8Ge_#2^wzPy+8ZJH&{Z6D1zcJ z%bB(hL(1n_?^z8qQ9qs1$;v-Be2*%O`Px&3z0IvQ2CZyj;&|h&2X&$HwRLK(!E|F2 z=8x~QpE(?oFf#h0{3o~F&%S5QVmwZ(NfrBO;ts986?KSreL;aYu4k(9;GBn_=-c-~ zQu>46+T=KoYZl}Zb@;hJzfR>1J0DwkG$x>&RGT#LfvTi1Mh|y?GNb8Lp73wExgN?k zQBjpt?^cC*+6iSdUiS{CTe%0#A>7YqK+@>j_`Yd;JiYJE(ZM^?Ik8s3L{L(8x+q9Z zAqn19q9*Rpp@g%Yx9+2 zw#huiVjS7ic_j%xR#A`Fb2d6b`Qs*)D^F{(OpKM@%s#u`1!0_WN%+p@_%IRrV!!3b z?h|)D#(e7{zEzP!C%+9kFhE3)LjL33Z#tHqhX=J zWmu88zUSn@w@cs7zik$7s>Rt)ep97{)KS0S&D|(|n)lL6?wLcmjT)R^E-bkhUDOl2 z(<>)#EUNST-3=Lzm~;}|;m@i2%kQ(9whl#hT~GuIlIt(6E?x# zYNuGet3TBCspqEG#?Uo9Y_jntSb6IXV-x2sotCGT=Qr)5du0c|!G-+x#OQdrO4+i` zXpX-~MsmTe;Tz(_sQzHDuY^g`Mx@+t?;m%WVfN-{?VfV3IAvDYZB&GMwlI7B!q^a} z>IatY?B}cj_Sc{6)1A0=Pl=;EwfT~uYi45i1&IR9B0IfBIi~)MnY)ddFMd*5!&b3N*$!ILT9o365BGZ`>5K@fpS;C?*~JWz zuDjhG%hny@pJtMBD-U5*tdYS7i`6T$IbW+$ezM_do{D#oV4YKnYUyur%TR}bFfgT2wF|(HeCNy-@)|b z;XKVmnx&M5ywcko(-Cw`z5UJ6`EpZt_^Z{0d>I{^I}Q?#UDNp{eI@BEXTKs3$q%=O8BkXYZ<|(I+geT$P#ow7KJPCLuK{7BNBS> z-Lgr&?97rbJ8ulPAHDs{vtxKUT&w?52!nfGx>OOs@1AmSdq$B=7?Y}N}M zrFOVH%V}{-2dQ*QxpVg}&#IPBrs^?OKX&YVqeK$ZZ}+`B#Khsq^l`2!mh(@bxsW@B zM{RLD^9q(feqkXj<}Dq!rEPO24C<)2PRkr~qyBzFf`(i5Wg)WR(sq(@DunzaV13+u|JJcGWp;6I=5)4%nNaw5cXN#XiZ)texm zx%ntlKmU-r7wNUS?uX%Rl9AVyHPQu3$}7cgROgqo1%5K59nNTzj1O%kHf(yvtWj`W zRe&Vc^}(V(Qoo~0`taqv{%4{wvr@0a7KiLv@=T|T%8psQyXH5c{uH6!%y;wT5v#hH zP_J8JUkYw#cD68cHuc}+x0k0~IpHFeCVubHd9i0?qqK(aS;>_3r1?Y1-9LSjA<4Ts zra>MV&DVDRzG57$@+UE7+>1nJZvra`z~C6G{f`R zPYO}(Cu`A`4U_N$n(Fp2Xa4MrM= z+`|fc$me(-M#64X?LGg5CMuarx5}PmGJy2necQalJd?y`Kl0Dx=H0aa_MXOg=K6!; z7BT7bj9zkjYCHr@dea2;KJ?M;L3aK#QOA|S*#?$<6YM#Qd&q9Lez%e3ruGyi>ZAMW z>B~$Vmr9_m=j~|u_!I1YWsd7HFLSHs_apaxG&VvVA?<5rhC&$L9g!4q@_Ez(rGLc& zTLUWVxUq~M-pNvCww#ERnh6-oS;<;>60ttFG;?LY!$66zOHXAp>EhM8+&4n2rDEc( zNR*QW$7t}uSO>A}hJ!^D=*_{UHDr4mJ5F3)F#a7BEL zy6{qrkH+`d9?^#xwsNk^?=vEmy*@DcMY<@N)tPFYE~W76f(`d7hU+BuJUM29f2{Q& zUD0dzW!%z>sgqCGWW85XP(9WBGTEMGgYyd+aa2%MikVx{?=KeJP0$)V9Q0CAIse&E ziEj$xz}1TSG{S6or1t{?ceX&|lWv0d28HNJNbCn2C3VW1J`(uoQ)Lk@`f-rga(3v% z21@Q3f*cKGOM%Qx>(F`aeei7MLr?JJrNc^(6xps{NPQD>$b9%{pVkG9lIa>gq~I4P zv7q3PryLor{*P3ME4gk)LaV*z38CG^X7fYzSOu&KNQavp6hHlAmEjPs-N}t z61f+o^Y{>r=@k7Zu?Ow(nQStxS7?J`9ICwA_bfGqQx5Bw$?=Vd=G-55ArE4D{D8i| z?d1OX#gvgt@?WN2;-?}jO|%<59#!@E>=!c~)J|5#{i2Lh-a|4cF!AJ&pTwgMM4O?D z*@XHJuk66*ZqzTV{QLtCFd-6sS*FcYOCBI1T!|AAs}TWLn%th_rsDS1yu7^_@x#(? zyqPc(|C=r034#U21nqC?K4nt+22M=G8SW>Id02FbG?=x6;1zsl#EtV6TIT>zxh(KAeOa?#A#jt}W;CWI<#7|N&)PiG0#}h}P z!;nEFZNggdr8#dVM|m4dDfT;Iysdec^1>%ds+EulKe#-)EPGXtx{8gmzMw8Ir`OuJya#K-6=cbW(>+|Jqm!;E#yu#ZTcdL+jt@Wh9p6ko)Wba!r)%_(SllCeM!H;u zC$ti^4_{Ti4HkLKx-i>`7_d?JVfBQ8O?f)b+>lO1@608cBb^By`KO1u7wBR3emI_(e>AtYe)s=VWDes5n*A6%8OxqnVPxh%-qTNailC7PgC zL)dykFt1=PX@E|ny{Y#M^Upm#w1uda>MxVQ&>yG!PQG*P3`?u`y_aTlM484rYI1pn z1VN5mxOvr^C&(|FedP{+1h>}NeCEvBJ^iMyn8&L1E!b;UWWS!#kGU;#$gN4V;bT}ror+GTh9r0lpx~a~HIP`j%x6SQaGvaQo>nlTpnxnb7pJ(1x zjF;4XE`7#Q+Otae@i0keq+A2BG1Vg};>9mD^_9`eFS?J*Or5wtN*gCn;V2($xI%&JypBlI$q`0hYOrR!tQ_`B_YFn=SJR}6Mi^dMo>$bJDUDZ@!2 z7xO~#CtgC1;Wyhtk@I{zGUp=Y5_9Co($YLPHpnCOK=cD(x za#ttG%VPX3-W{ZSedF7Y`)6d`?oKw#JZ+7(6V8h$n|6LcLV-H{xGUJ$mR|eE7Gj<1APQ{$7U3p;3X$V!)nnKOA9dt1)8Jmd?s23nI zsFg17YPTql)nDVtD{2?NGWc}R)r&f~59MItZwn0-ZhkpEnM|cVzsw?lE0*~-foSj% zZK8@K?4g479cYI3W5mJYGfr^~Syy1B=7&jcX~W5(PvQ)ogLn?Uay>yyD z1uFE+1Kgj(gGv<&RW$i?FO^hMee_xv#*%cR(kf0lkI^ZlwsE3drfJN0F7Dxnuf5!~N^-;U{ur^y*rH zAYb>q5YdTyvr7YRp|@{MH?`!vW%1<0E9p(ld(od;yL7)C_lbl<(bcXPqV|(W#t}E- zYWU!k_H$gx7bzcmQ#%jjm?~ITA0W$rbSA#hSKI3F7m^DMD$fjR)yFD`w-iM=Moz; znWTshq?2ZKHy*|L-cZLK<({9yB2C5W$RGZ*AYWn(U!8_`UsNs!UcX6kG9eS)WpOFlI3-w_Go`Iq~z;){NMuO#pe_)JHc-?llt}T6l?u0xgwWy*tMSYjm^-X zO-**yVE%Z-befd)YBuCO{cYiUjx(M7W@1cLrFW3TK8EB9joovL9=WG72N&2lsEahF zS2@pI&QobSbl~C8(3_K7h4X}!2*T6lf@HK6Q}>Q03!e5o<>Snxq?GvP>GZ`h-XNn( zmS2`g`b#@m%1R!$Lc&WPO;wE2dc_~Tw+yR@Y%te1iH}=7*P%Wro}+S6L5<@hWz$KA z)gaHK7b03rxMg$jTH9Pv#>r=wg`Ipmy1yXx_i9G;$oerQKip%?27RTFXXlDYIB=M2 z=pj_hobHLG)Z~=bT?jIepDF zRJC#9O7PbYM*ciM3Tc~c$sSw}3XM$bBuPE=rJ74T%K0KHNy)3+I=7TWUq5?5ulRkk z(M!39Uo4@c?V6sgd;^2o2IJLnW6?_kO4-e&@y={iZvx1kmgD{0U;5|(^(Z~eHo>p- zCZZ^nQ@J=QF?jDNWzoT&eB-dwCj{QRC%{#oL7Wu0VwRz;vd*QYhVdx{fQ|lh|4RnODL(U%Yg}w z=qZ6FC}aRF0hUT!`x`A`gHXk=vv;Q@d7 ze+Qj*8ir1kvopAAW+-N5a@7bN0Y9K_XJmN+%*la3HGh>t0(*U^gsFiodP?Ofr~&jc z)DUWDebw3udIf3(HG^6}uR^V$*3fHETc{n>0qVrMQ9Uq?(FlTEkD`U1mbp3o^0!a7 zHXK-Zq5px1wwwqEFAt292ZH2*b0UBJ<>7{Nf+?54MV$x9&&dzrfpdc#yj)0-o0|`T z{>#n734{E~$HxPXzuS`X*hd5IM_uI@CWrrpSxG`V`m%*-9wE{GX=C{^rtOj6($ntN{kCfOb59o;#S%yS1|aQ(du5VcVSK219Ufoz8|(;lg>7 zXvbxgK+#uiL}YwJRu@GgA#8{;+?>$Mm{9OAE;&6tOkDP`if5sBPF!@dx45jo_;YE` zl+3U~X?J5iIrFfTOqg$m*_DT?5qGq%B&S_+@aYt z1ARxiz}bGy{zo^b*j5Jsj8JyAMh2F|j%}a~QQ<=3T=q^p=Z)mVp*@dJjEyUpdAa}p zUijuUuOqPa;{QLoaLn7+`Yncmh1Qn8Oj#LiI zjSmlBZ&yBI>EXUoQh3mM&AmWFu`n2sy5wQwzC1V|@|9Q4X4%~9!0g=OH&kh0+lQJE z(fsbx#RVtH#PRu%_xBH6dZoFS75P552qX>%aC;xhr=Q1pP!nBPGfx6(KPT9l@{&1M z@Pj}Bi{H|l_94$7!c9ER`h_g*(&ShB&*rbLl|_bGJAM386lZ-zIJ*Pd%GoW5~y1=cwBOxbq_)PoI@gE&^0xxBl?a%e#$2ezN+Jy{+S-8LFc5+Ar z+qdIJZHhV{)}t(wg6luUJJ<~S)1Dr>pU}c9dnAKyayi91iMt__aGp88j!G_*y;)kJ z&*ScsS(p1y*3w44+eRc$WGu*w&1A8DQuQh2WDQ2BM<<;c`+**V?b99T{J}~3xRjC&`H$2hWF|T%{ zGgt8^@$))-!F)2GRM*AM`^WE7hs2eKDEU;5SM;-9qT4Tj{76J#3S)?pOvO)*>0d`0 zOOMQ-&YLV=X47j;3gn%57|3g%)hp<7sq#qC!I{_+XkU<$aJP?qSBub0mGa{8Jl9tT zCoP-|C2Hm$M)6*7E*&+mwi=W2G3O4sq<&O2BXZhH<}-Pm0<+bZTgw7sV%Gdcna9$G z){6XxW6nOHy^c8B&wZO9@+8Myf=HI9PtBckqEgQ*vGCfs=MlynRu{^aUG-lyT8qif z9=k+ck>){nRs|o?@pKPVBP{gt?N74sa+*ic1H}@Fbitvfw`XOM@mZQ@63qg`Ob=6^ ziWxX@KnbnHy zg#=T71u}4on~7%C5+D_v*$@=i18PA8YK)i2>|p7}ieE>1;fCqqPh@_^eBz08)r@td z2f@V~GR1t`<7ZspD~ZqnhdCMRR`P!I-;`9>v;h^_dNttGETAq@;M7#0E_C4ZM&!zQ z=JF95?@) zRg}YA)19D|@IZn4f=KGs zyjeG;N7OmXS2e%) zxed!d8*@@RPBMu!M+~2RdfM1QQpxyW9KG9*GwG|Pi(~3X_kQB`er|t0#g=NO{(_CV z@GxbN_tz8trmy!fJ|0xGrISW}J*VE{oh*ZFa!h%)CzY*6Eb{BQfs2s`UF{>Bg9m0l z$EnrS-Mnf0Twju3i|h5#9|ZAk$ze&jB%f8ia&wWk$gfWob*wsCwR?&?4&)ZSH7aB^ z_K=ITeqw4Lbto!~vfuReEl^mRUn{!|UD)!Z-b-ubvSRV~Zd0C^D`sVNW5^FPCG=H# z;ZJ`ywOZauPL;{+M*2t0oHylpARPwRzfIXBAzBsL;G6ck6?K@hiJG-4rNP(q3^^n1 zxXa!{bvH;A{7Z#$XOZ&$xQraEgo@dZ4sn$qts;PO$?UHtYi9bp5N8){vfLFQr@tW(AV_ z#xnl*j89iZ9)ON(#R#+N9nMDy9zX z|K5e8RTBrGReUkyta@!W$Yu3s-+N7Y(ylC)-Zb`+k37y_b^I=e+r=1Nwu{c55eD&f z?xaR7CLBb|fjlIq%0&#tT1>_ovuiXd=IBV?Z0ugvkB8}dx>ejnAMQdJ!w9+%w72F1 z3BCy%z9vE;1LU}5@QtI^?~Z2|za47$oudo|lJcH%Os>sPtz`$ zR9zxIs65K&%vIyKVD%=Km8Co<{Mz-N;3VzYz+{fkh&6joPX6bwa)e2rQJBng1m^y@ zH}y``LPnYJaOySLoUfoBhnqq8z(cvn0u@rvrI7)j*KP8JKXWn$s-0))7i~u)vAiH{373g^ zK{7EShDrbl3zOF7!fsM~=houDN8y-p2%Py!63 z*s5PMFf>|!`&E%pM=x95cm;((C5`OO42{%e#C6fHi|AATNPqc+xl@n zFskxX zqXuo10b@d&H2_z+9(83S1wH<=c>?s1(B=*pW7(_$0AStqf4vMC&;SELziI%`Sa(g| zNI{QpZKfd618l#uaYNQ!kvFpOK-OJ*H+FczsMTgRKFGT3=|&0|&)PfzKyckPay65KQxhLRvy6bwU3 za1UV^T7uX{FbpxlErns2i5}^}P!rr?7>1i5atsVZPH@9v7!TpC}_z9LK zg0YF;m;pM#FboA@Bw!ed0(QVK90gGiU>K4DvcNDb1t$G43{3%dU>KePwF_2F{{B5c zB^ZXOzyckHp(7>2fhvoH*AftM!?LtL2swI1IyG z;FJi*a2Mo)W4H^9{ookxf--OncYzNb9K&5S{oxpb!hmra9K%rb;{eA{6fGFx7>=SJ z4>*RTU=SYcZq^@Ap2I3&2huwZ(5x89l44Gjd za4!PGW^{9dz|a{6zEmK#@EP1p1g0^DfzYT3Olyp8ju2be4%%wO7Pf<1jo89=&{`w5 zupQia#1^)LXo-j|YzIU@Y+*ZSu@PI?4tRms!gkPRgD)AI*9&NZ*ur+uY9qF=9k2$m zh3%l-Mr>g_Akjw1?e)$F23l^!7PbRUA-1p`wB3j;YzGuWY+*ZSy*GlBZk!4*ZX3U#Xy0Ti6cRx)Dz3S1PF0 z7PbTOZiHmnNChqVM$nL7yP#HE*bdtAjlc`PQbDb@upPAK8|L-DQbDb@upPAL8wTLN zQbDb@upPAM8z#ydsUT&;=J(ewsMQv>gU14~h3%kK-!Sw1l^Kk>ZDBiT*Eb9if2D$V z!!2wFE&GP4+^k?%s|{4n8g6vgLhYOnS4k% z7~hq!wZ67~9LB}T3;2ruu!#V-hMOOb{^ZFAKJ{_&f_!4OcKwX141)8RHrer@bLZ~U`yl^4-1mM>?y|yWM`a zj&(6N3AAV!O(!~7v^#ai5!76aBX_YM&Y!5=?{;dRY<6LE!wn9;K{2bCSqT@n+2Q7b zW|M*p%0LOHQk+lsu3vDY2-N+d%FhrWJFgr4AV6QN0d*`|&wF55!Q% zlV^^dv6s0{2b|#{>Vw_1sm)Jyt>TV+ZW^m|ryf6H{)8(>sk?@GrAqaFq2-)Hv-?i_ zyYdHgQI6$6wAlBqRc4P5WT46fVMhRV$eD2AZ zQ49ZZ4-_4P)Ju+lpT`~fYnU=TE)u->Ks@@^+jBfe#Ytaqq{I7laci$)_iM}U(c(#a zktvsv=DW9Vv*e$gI*K1c+j@cg5(RwNs7xeM!9vj~Un#skXoyJCWhv+#LWq~^i1NG9 zq;J#YPY>pnkC}A3+dBnw;yeG;|3Un`%^LrD?8*4t=MMK(n_lW`>5MOMc{XkShvM@(zYp3mgYWKj6;`Yq zh~Zr_IC6se+o7}*^k4NP2gE!s=ewMCa3XINhs`FWe!g~%V@NjpgLaM3ff?k5*lx@2 zAf>QnN-sZOCBk9x10;R4wS(n|NN)$> z*FBWHK$mNfGh+Al>Laee@sk&m+u4Sl@+i7+VQO(|Z%sPpC!aR|@H2|bXl~-NBsqVH zT3cgUHoe90_G0dru;32+=DAZ&lq5B+`=7|SC0b=J%bK-&RajlcuhbG{UcT;O@b>Eb z%#8AM)=3tY+fwI$>^Cc`i5VIH+M>5d5#J!wu=>~+(R}$=7fj+mlm8g-?X;FC>aK44 zxk&G^Y*ABf72J|>X?bF-*|#YR|H-8q&HaA854(=$xN!QLYo8Mi@i}gYw10cU>iF0N zi>6O&YTZt^pVK#-Pw^oppIg@MT(0DheWCnP-6LPu@a*?8Q$ziuPG&FnTD6s;+^hMW zxPq()-7-For5`LMvTrI4x)c~KS;4e0Eb2^JxLC*?RH2w|ezmDA-$|364%K`}ohN(X zY9lv)85gRl6a_b~CL8!x78P_@z3vdb({ZJ}f_)N}dMCarf@8s+zAG#1=-zcHV2*q<-IN8*xe$3eq@hrA62rvr4KeSZ96j_qf<@(d%b ztNv>lj=r+XV;m31v~T6f-5qAFe|$`Q(Aa{u<;;XJDwa{0M%E#v!VB7}p4Y6e9KxRK zw#MZ?m)2g6_cGSx>1U0&;^e)FaxDUSo2A%<_ie-w4#S!YtI~dFZW}vE@ru(7(-D1n*PQpcx}nL(q4Nu>VEMi#ff~6C zPYR`N?``j=U*LkTGszJ-_I7q2)M13qb~s~GkLbC+IL%zMwN~|gAI|9Z`X`_NT^5;$ zC3z~N(r1!1)BUX9BU4-2Put{kRH0sX70)r~Po8_huGG}!YIb6NKw+Op>4}ga%PTWo8~nh6#)8B5UiQCZZYY1*mTcx^ zldhh=$4t#kOGN0lB=We@C!fks(ua+EY(FNEh%?fCru#;c!ZIBE0`@wyR(C0olAl1M zWPFTt9#++kt5xhv6HhSO7#Q|FjO4?xBDtP}F-s19<<-2rwLu2%CnLGbK0jy6wa8!Q zpU+v8Uaegd={!6%cp?F|M3Kh0hqKP7>?nh*zsQ~mA~EkF2?|}JB=Y@;{R?GgAIeI) z&E$I9$`1ufl~z1{uyD8@HDU4eV{(>|O+UNx&4t4aBqQv~Z@Tx?wZAWLWI zyexLCz?*h#4J7EPD-UFTBWe^D9A#XLtLu0izRGC6~-H$;4l;cX_x z0hvR1%)$Fyf}*7vPJBL)efB0968~vn0)7>)1;fx8lCqJ#0<|-4wUi%^sJ?EYKR84o zyHKDab0)abTY#`$fUPkQC;I3?b7I4p&=YSy-%t!DM(&04v(@6g?@K$Q00~dR%RnEr zBf4UHE`jk>c#`-riRh!$=4TVYQEqRLwUM-a@1N;5=+ z-mg?SS$Ru~0(}Ohhv>3i=(0yE`h=u!T?BQYG2Ms$37XIks-9%%bJ^`d?;AqG4e?Iz zi};}WFh%90jCYI}$ZAezWP2`+An*fSqQkNp`Zr7o(3vIHe-jt|n}^=_MV(>CLmy>D z7dBom4606Jj*BkL9Qa`$C@jC8nF3uwnKLJ42>wr7*B(_>m4~MWXX5H=5@oWQHQ@vF zQo((lbM{f2%m+n)0b+*JHHixeHX`87rde8APKjs=I$&CnPcmiF)RpM6Vp^hPNon|4 zqSI-Ts-k(#jWa`Kzp2|LJ%QNc-Ut3t7SMRNUFQ=f`cl|vJ zR{qip(XNTi%xg$o`cBTg2S!~<>oq*)quI}&oin%pyJv5S?;4QSS?_C&N$YI%O&Bm| zZOg%_7Z-bi&wXSiAH4S7PB{c~DTdhJX_V=Fstj*Zhx<|+08>gn|m?i+j*!EKS#5=2EQ5{S9&u3@LS2>RxjPR zXhrEw&EGd2Jmk&KzWK$UPsGi+w`;-p9hW{l*g9#*lue`3*UrWT(B{6!()X7R-O>2+ z#?8S;K7M4|_zN8)b`Sb$kY{_qw#UH;%7$`j5S`43fXT3vpA zpopIsyYH3uhu1Zpn9^&{z?l~&zH@O*OJ&ujC9gki#?+k{@Of?BEvGBKsVE$|de1)x zypWV?b^NRMTQLRKz47DFMA7lUkK6mVu9-Q%rZ5~XT;ok{h%WZ*c(%hxKH`~tv@v@` z$EBFNy}vp4#Hqw#`})>pJf1nT`kCA7&ety;&^j^Y_|~0G zJxinSud2;W7(C#qmal6PzN|p~^Sf?7d?-FVC=zbUd38y{u6-AuiL2|AUpGCoF&v1T ztUA6tvcEb0P49uF(c6k^YX&aLzHe_Vs>)dSSXFK$j5-%>e*H`D!e!Cu;)NfL>$7Eg zUWGqKwr9e$FlHngS#oiRy&a|hw!JxR!Q|0j-W*@rD;&Eo^khYPP3-cj^C?+@dAT(w z6YopPF5R{}??7|2{OtgK`{BKmeEimMEHNV~JGuYv2!4CH=Ju7NUh6m)m(*|AXT!s7 zk!#ma*_iR76r!XycKL~DN;$IZ`EdRCr0h*ccbz#9({|VOr)!4x8@Av=_#31?`}d5O zGe7Mwp50rvx@J`VWzS=|o}!pjFI*XO_WiWHM*{7$7f*=uPOrPPVO?%V(}k?`4V%NC zuByq*|Ks7t8Dq~*Ps?j~wd|&xd4Js-SzVKzKVs6@=-|;CtLjI!wyzwN+}^$r|LODK z;R(67^_w-nqGrC74R{lJ2(tn`6 zeEsN^=N>xVx_^be?aG*@^JC7mbq#+hFep%e&l}5gHypYlHha9cw)duKb)QUoH~rX# z6?cdcuN)ugofMt?N^REP-oEqvw?7R(zP|JEaXYts92nNNar=*n?dPuhedoctF_#y3 zx89I*(Z9@dXIpVg<+_#E2Vbi)kF@T2aBuz8KdtRO>ESOXmhOG?r7J(At~zv}fqg?# zt#YeT*#(alRZ7=T6k3NZh?d+1;x7=f;4Plft5awGx=OCFbQA^AhD z6Fiw!7AYw%DT)jmI~*FS&u98MlSC#5D@R3L+phe5UkDw!GVJC4(*^IGL ze7CTM-#|VzMDv?$NDl#?s3VRe0o*M zYs$%`V79cxxiQ$=;Wtt+M1eB~zlWO6l%_2uFZU_t1}vsZIy6n~Q<$WsWz=0>fcw-R zuviuZfKz!v$Ajc94+d1K12#u$FHDOhGpupce=xdHAHYsUJi}i)-5V-L`dF9OnwLDvz zy4~g3KIXI?WG>i2T3_IKlXhpNJWE>UT^N>P8PuR+^1UYobBw?WhFc$$4);L_6EZs1 zofm#EwXXv3V5#E)_nv$SQ^{+<_n~4!Is=q+wse=cu|PmRXrN-WCI)c1gZc_iFRCM8 z(^>-8pnO%xT?afU)K_qnP#u7%ui!$U@qpEx`U&u;2_%xXGO{?jgm#LJz@>BL|I$kGn%8?g8+B_czRXny0|zvy`rSgFZupnY2D*^8$2x97zeAqtJ@qlwjgE1F^L4!C0jJN=2tOjE)z%5Js2;zbl$rmzp7;zyOvKcd& z3%D0Y{D32u_<^SBFyaUNx*CkR01q$m0~m23h~p6T0j?*~SXzD!rs0BxdyXm>EDaYd z!Thi^T)A;w@(ELiF&Bi*n62T0jgeE^VQaWxTg(rexd0zC@dFPd5*OfY)?my9 z_??L#z=#WYTS|j57w}RK@dN&8;z9_)QtEgBBQ8iaK<&R!fVlw2HSq&?b%_h0stzMA zVAg9e<^rzP5I=ws7hpQkV9W&=4Tv8&mJk;(x`Y`4CpgEoe;5dLQqNMr=p1XHo7J-v zY9*Zn>JjyyRGBC*B2?7#sUa6Jg$q!vWC)F0=Ae*q7H$laLp_HXAvA>2@t$npIIhk$ zST{�wy2Nb=PYINv|~wKA$4MK?ZXvc`ec*4AaN`41JK=h9fn}4E2cm&$Q@_ZkPzd zP-U^4h)%bB7BmY=2SXQQGP+F3D@dm^ESqdbh6rddCtS;2o-j!#G4Nsrjjym7v$5Ez z^4U(rl)Dbwd5%!QEYced8~4?f@=TvkgUR?fS=|3@c(80bVe|@S(D-6S)M3OU6S0j{ z2f`3ZKEI*Ei07tXXfT^>GWeQ+1`CqDWg`BOxl$=#>W z{kS|F2GsIEBgUL4Wp{a4WU0K6Az$ZG(g__#TrdqMZrGjIU@n*j+gVM6xnLTsvjZb8 zn9%)5W|O&qFG3J!@T~~q0w5j6T!8LJ{ReXx$pYPvF%1_4uZgD6a6zz*%oG|fK=&j0 zgoX?FKnBTYYq($=%nw_`1>0tR*vthBx}Pc^PA|j-%ZE`{=A`9gE?Ch0s62QbC@;*r zj4>B1KiffYVkIufErrTMv^mKF-H$Qmf(6}=%VREBK_}qd#W!HY1?` zC_U={W~jO@FgRk}vfy-u&SFgUEy`1vYI^`ib^&0vI-g-^!D6S@feT2AT>ve^;YUe_ zcmdUi2G62%e`sS`?`#MN#Dv83jfmCeH6U{k;)@6_(F d8%TzKIbZ52sF>x%yr70)$nQx^9Cv@3=f8wj89o32 From bb4ea60fa80b6948eec136304f0f5887746c4977 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 7 Jul 2025 17:41:33 -0700 Subject: [PATCH 239/277] Update bias paper reference --- docs/source/faq.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 43fea0c87..998a5bba9 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -1316,8 +1316,8 @@ resulting from |zfp|, as detailed in the following publications: "`ZFP: A Compressed Array Representation for Numerical Computations `__," International Journal of High-Performance Computing Applications, 2025. #. A. Fox, P. Lindstrom. - "`Statistical Analysis of ZFP: Understanding Bias `__," - LLNL-JRNL-858256, Lawrence Livermore National Laboratory, 2024. + "`Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm `__," + SIAM Journal on Scientific Computing, to appear. In short, |zfp| compression errors are roughly normally distributed as a consequence of the central limit theorem, and can be bounded. Because the From 5fd66864a33dc6df9b4eba0d169eb5aff31b64ac Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 8 Jul 2025 19:40:50 -0700 Subject: [PATCH 240/277] Add detailed documentation of build configuration --- docs/source/bit-stream.rst | 2 +- docs/source/conf.py | 16 +- docs/source/configuration.rst | 399 ++++++++++++++++++++++++++++++++++ docs/source/defs.rst | 1 + docs/source/index.rst | 1 + docs/source/installation.rst | 33 +-- docs/source/word-size.pdf | Bin 0 -> 47962 bytes 7 files changed, 434 insertions(+), 18 deletions(-) create mode 100644 docs/source/configuration.rst create mode 100644 docs/source/word-size.pdf diff --git a/docs/source/bit-stream.rst b/docs/source/bit-stream.rst index c77d2ce36..f46e6ae23 100644 --- a/docs/source/bit-stream.rst +++ b/docs/source/bit-stream.rst @@ -14,7 +14,7 @@ against. From an implementation standpoint, bit streams are read from and written to memory in increments of *words* of bits. The constant power-of-two -word size is configured at :ref:`compile time `, and is limited +:ref:`word size ` is configured at compile time, and is limited to 8, 16, 32, or 64 bits. The bit stream API is publicly exposed and may be used to write additional diff --git a/docs/source/conf.py b/docs/source/conf.py index 0536c39e6..52764249a 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -30,7 +30,14 @@ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['sphinx.ext.imgmath', 'sphinxfortran.fortran_domain'] +extensions = [ + 'sphinx.ext.imgmath', + 'sphinx.ext.imgconverter', + 'sphinxfortran.fortran_domain' +] + +# Ensure rasterization of vector graphics uses sufficient DPI +image_converter_args = ['-density', '300', '-geometry', '50%'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -137,7 +144,12 @@ # Additional stuff for the LaTeX preamble. # - # 'preamble': '', + # Unicode definitions needed for TeX Live 2024 + 'preamble': ('\\DeclareUnicodeCharacter{2212}{\\ensuremath{-}}' + '\\DeclareUnicodeCharacter{2264}{\\ensuremath{\\leq}}' + '\\DeclareUnicodeCharacter{2265}{\\ensuremath{\\geq}}' + '\\DeclareUnicodeCharacter{221A}{\\ensuremath{\\sqrt{}}}' + '\\DeclareUnicodeCharacter{2248}{\\ensuremath{\\approx}}'), # Latex figure (float) alignment # diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst new file mode 100644 index 000000000..3e7da15f3 --- /dev/null +++ b/docs/source/configuration.rst @@ -0,0 +1,399 @@ +.. include:: defs.rst + +.. index:: + single: Software Configuration +.. _config: + +Configuration +============= + +The :ref:`installation ` section describes compile-time +options available to configure the |zfp| software. This section provides +additional, more detailed documentation of the rationale for and potential +impact of these settings, including portability of |zfp| compressed streams +across builds with different configuration settings. + +Unfortunately, |zfp| streams do not currently embed any information with +regards to the settings configured for the stream producer, though some +settings for a given |zfp| build can be determined programmatically at +run time. We hope to rectify this in future versions of |zfp|. + +The following sections discuss configuration settings in detail: + +* :ref:`Word Size `: :c:macro:`BIT_STREAM_WORD_TYPE`, :c:macro:`ZFP_BIT_STREAM_WORD_SIZE` +* :ref:`Rounding Mode `: :c:macro:`ZFP_ROUNDING_MODE` +* :ref:`Subnormals `: :c:macro:`ZFP_WITH_DAZ` + +.. index:: + single: Word Size +.. _word-size: + +Word Size +--------- + +|zfp| bit streams are read and written one *word* at a time. The size of a +word is a user-configurable parameter (see :c:macro:`BIT_STREAM_WORD_TYPE` +and :c:macro:`ZFP_BIT_STREAM_WORD_SIZE`) set at compile time, and may be one +of 8, 16, 32, and 64 bits. By default, it is set to 64 bits as longer words +tend to improve performance. + +Regardless of the word size, the |zfp| :ref:`bitstream ` buffers one +word of input or output, and each call to :c:func:`stream_write_bits` to +output 1 |leq| *n* |leq| 64 bits conceptually appends those *n* bits to the +buffered word one at a time, from least to most significant bit. As soon as +the buffered word is full, it is written to the output as a whole word in the +native endian byte order of the hardware platform. Analogously, when reading +a bit stream, one word is fetched and buffered at a time, and bits are +returned by :c:func:`stream_read_bits` by consuming bits from the buffered +word from least to most significant bit. This process is illustrated in +:numref:`word-size-fig`. + +.. _word-size-fig: +.. figure:: word-size.pdf + :figwidth: 90 % + :align: center + :alt: "bit stream word size" + + Top: Bit stream written as (from right to left) five sequences of length + 12 + 1 + 25 + 5 + 64 = 107 bits. + Bottom: Bit stream written as 8-bit and 32-bit words in little and big + endian byte order. The two little endian streams differ only in the + amount of padding appended to fill out the last (leftmost) word. + +Determining Word Size +^^^^^^^^^^^^^^^^^^^^^ + +After |zfp| has been built, it is possible to query the word size that was +chosen at compile time. Programmatically, the constant +:c:var:`stream_word_bits` as well as the function :c:func:`stream_alignment` +give the word size in bits. One may also glean this information from the +command line using the :program:`testzfp` executable. + +Unfortunately, |zfp| currently does not embed in the compressed stream any +information regarding the word size used. If :ref:`headers` are used, +one may at best infer little- versus big-endian byte order by inspecting +the bytes stored one at a time, which begins with the characters 'z', 'f', +'p'. On big-endian machines with word sizes greater than 8, those first +bytes will appear in a different order. + +Rate Granularity +^^^^^^^^^^^^^^^^ + +The word size dictates the granularity of rates (in bits/value) supported +by |zfp|'s :ref:`compressed-array classes `. Each *d*-dimensional +compressed block of |4powd| values is represented as a whole number of words. +Thus, smaller words result in finer rate granularity. See also FAQ +:ref:`#12 `. + +Performance +^^^^^^^^^^^ + +Performance is improved by larger word sizes due to fewer reads from and +writes to memory, as well as fewer loop iterations to process the up to +64 bits read or written. If portability across different-endian platforms +is not necessary (e.g., for persistent storage of compressed streams), then +we suggest using as word size the widest integer size supported by the +hardware (usually 32 or 64 bits). + +Execution Policy +^^^^^^^^^^^^^^^^ + +The CUDA back-end currently ignores the word size specified at compile time +and always use 64-bit words. This impacts portability of streams compressed +or decompressed using these execution policies. We expect future support for +user-configurable word sizes for CUDA. In contrast, both the serial and +OpenMP back-ends respect word size. + +Portability +^^^^^^^^^^^ + +When the chosen word size is larger than one byte (8 bits), the byte order +employed by the hardware architecture affects the sequence of bytes written +to and read from the stream, as each read or written word is broken down +into a set of bytes. Two common conventions are used: *little endian* +order, where the least significant byte of a word appears first, and +*big endian* order, where the most significant byte appears first. Therefore, +a stream written on a little-endian platform with a word size greater than +8 bits will not be properly read on a big-endian platform and vice versa. +We say that such |zfp| streams are endian-dependent and not portable. + +When the word size is one byte (8 bits), on the other hand, each word read +or written is one byte, and endianness does not matter. Such |zfp| streams +are portable. + +.. warning:: + For compressed streams to be portabile across platforms with different byte + order, |zfp| must be built with a word size of 8 bits. + +When using the |zfp| :ref:`bitstream API `, it is possible to write +up to 64 bits at a time. When the word size is 8 bits and more than 8 bits +are written at a time, |zfp| appends bits to the output in little-endian +order, from least to most significant bit, regardless of the endianness of +the hardware architecture. This ensures portability across machines with +different byte order, and should be the preferred configuration when +cross-platform portability is needed. For this reason, the |zfp| compression +plugin for the HDF5 file format, `H5Z-ZFP `__, +requires |zfp| to be built with an 8-bit word size. + +On little-endian hardware platforms, the order of bytes read and written is +independent of word size. While readers and writers may in principle employ +different word sizes, it is rarely safe to do so. High-level API functions +like :c:func:`zfp_compress` and :c:func:`zfp_decompress` always align the +stream on a word boundary before returning. The consequences of this are +twofold: + +* If a stream is read with a larger word size than the word size used when + the stream was written, then the last word read may extend beyond the + memory buffer allocated for the stream, resulting in a *buffer over-read* + memory access violation error. + +* When multiple fields are compressed back-to-back to the same stream through + a sequence of :c:func:`zfp_compress` calls, padding is potentially inserted + between consecutive fields. The amount of padding is dependent on word + size. That is, :c:func:`zfp_compress` flushes up to a word of buffered bits + if the stream does not already end on a word boundary. Similarly, + :c:func:`zfp_decompress` positions the stream on the same word boundary + (when the word size is fixed) so that compression and decompression are + synchronized. Because of such padding, subsequent :c:func:`zfp_decompress` + calls may not read from the correct bit stream offset if word sizes do not + agree between reader and writer. For portability, the user may have to + manually insert additional padding (using :c:func:`stream_wtell` and + :c:func:`stream_pad` on writes, :c:func:`stream_rtell` and + :c:func:`stream_skip` on reads) to align the stream on a whole 64-bit word + boundary. + +.. + [figure showing overread and padding for two different word sizes] + +.. warning:: + Even though |zfp| uses little-endian byte order, the word alignment imposed + by the high-level API functions :c:func:`zfp_compress` and + :c:func:`zfp_decompress` may result in differences in padding when different + word sizes are used. To guarantee portability of |zfp| streams, we recommend + using a word size of 8 bits (one byte). + +On big-endian platforms, it is not possible to ensure portability unless the +word size is 8 bits. Thus, for full portability when compressed data is +exchanged between different platforms, we suggest using 8-bit words. + +Testing +^^^^^^^ + +The |zfp| unit tests have been designed only for the default 64-bit word +size. Thus, most tests will fail if a smaller word size is used. We plan +to address this shortcoming in the near future. + +.. index:: + single: Rounding Mode +.. _rounding: + +Rounding Mode +------------- + +In |zfp|'s lossy compression modes, quantization is usually employed to +discard some number of least significant bits of transform coefficients. +By default, such bits are simply replaced with zeros, which is analogous +to *truncation*, or rounding towards zero. (Because |zfp| represents +coefficients in *negabinary*, or base minus two, the actual effect of +such truncation is more complicated.) The net effect is that compression +errors are usually biased in one direction or another, and this bias +further depends on a value's location within a block (see FAQ +:ref:`#30 `). To mitigate this bias, other rounding +modes can be selected at compile time via :c:macro:`ZFP_ROUNDING_MODE`. + +Supported Rounding Modes +^^^^^^^^^^^^^^^^^^^^^^^^ + +As of |zfp| |roundingrelease|, the following three rounding modes are +available: + +.. c:macro:: ZFP_ROUND_NEVER + + This is the default rounding mode, which simply zeros trailing bits + analogous to truncation, as described above. + +.. c:macro:: ZFP_ROUND_FIRST + + This mode applies rounding during *compression* by first offsetting values + by an amount proportional to the quantization step before truncation, + causing errors to cancel on average. This rounding mode is essentially a + form of *mid-tread quantization*. + + Although this is the preferred rounding mode as far as error bias + cancellation is concerned, it relies on knowing in advance the precision of + each coefficient and is available only in + :ref:`fixed-precision ` and + :ref:`-accuracy ` compression modes. + +.. note:: + :c:macro:`ZFP_ROUND_FIRST` impacts the both the bits stored in the compressed + stream and the decompressed values. + + +.. c:macro:: ZFP_ROUND_LAST + + This mode applies rounding during *decompression* by offsetting decoded + values by an amount proportional to the quantization step. This rounding + mode is essentially a form of *mid-riser quantization*. + + This rounding mode is available in all compression modes but tends to be + less effective at reducing error bias than :c:macro:`ZFP_ROUND_FIRST`, + though more effective than :c:macro:`ZFP_ROUND_NEVER`. + +.. note:: + As :c:macro:`ZFP_ROUND_LAST` is applied only during decompression, it has + no impact on the compressed stream. Only the values returned from + decompression are affected. + +The rounding mode must be selected at compile time by setting +:c:macro:`ZFP_ROUNDING_MODE`, e.g., using GNU make or CMake commands +:: + + make ZFP_ROUNDING_MODE=ZFP_ROUND_NEVER + cmake -DZFP_ROUNDING_MODE=ZFP_ROUND_NEVER .. + +In general, the same rounding mode ought to be used by data producer and +consumer, though since :c:macro:`ZFP_ROUND_NEVER` and +:c:macro:`ZFP_ROUND_FIRST` decode values the same way, and since +:c:macro:`ZFP_ROUND_NEVER` and :c:macro:`ZFP_ROUND_LAST` encode values the +same way, there really is only one combination of rounding modes that should +be avoided: + +.. warning:: + Do not compress data with :c:macro:`ZFP_ROUND_FIRST` and then decompress + with :c:macro:`ZFP_ROUND_LAST`. This will apply bias correction twice and + cause errors to be larger than necessary, perhaps even exceeding any + specified error tolerance. + +Error Bounds and Distributions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The centering of errors implied by :c:macro:`ZFP_ROUND_FIRST` and +:c:macro:`ZFP_ROUND_LAST` reduces not only the bias but also the maximum +absolute error for a given quantization level (or precision). In fact, the +reduction in maximum error is so large that it is possible to reduce precision +of transform coefficients by one bit in +:ref:`fixed-accuracy mode ` while staying within the +prescribed error tolerance. (Note that the same precision reduction applies +to :ref:`expert mode ` when :c:member:`zfp_stream.minexp` is +specified.) In other words, one may boost the compression ratio for a given +error tolerance. Viewed differently, the error bound can be tightened such +that observed errors are closer to the tolerance. + +To take advantage of such a tighter error bound and improvement in compression +ratio, one should enable :c:macro:`ZFP_WITH_TIGHT_ERROR` at compile time. +This macro, which should only be used in conjunction with +:c:macro:`ZFP_ROUND_FIRST` or :c:macro:`ZFP_ROUND_LAST`, reduces precision +by one bit in :ref:`fixed-accuracy mode `, thus +increasing error while decreasing compressed size without violating the +error tolerance. + +.. warning:: + Both producer and consumer must use the same setting of + :c:macro:`ZFP_WITH_TIGHT_ERROR`. Also note that this setting makes + compressed streams incompatible with the default settings of |zfp| and + existing compressed formats built on top of |zfp|, such as the + `H5Z-ZFP `__ HDF5 plugin. + +For more details on how rounding modes and tight error bounds impact error, +see FAQ :ref:`#30 `. + +Performance +^^^^^^^^^^^ + +The rounding mode has only a small impact on performance. As both +:c:macro:`ZFP_ROUND_FIRST` and :c:macro:`ZFP_ROUND_LAST` require an offset to +be applied to transform coefficient, they incur a small overhead relative to +:c:macro:`ZFP_ROUND_NEVER`, where no such corrections are needed. + +Execution Policy +^^^^^^^^^^^^^^^^ + +:c:macro:`ZFP_WITH_TIGHT_ERROR` applies only to +:ref:`fixed-accuracy ` and :ref:`expert ` +mode, neither of which is currently supported by the CUDA execution policy. +Therefore, this setting is currently ignored in CUDA but will be supported +in the next |zfp| release. + +Portability +^^^^^^^^^^^ + +As :c:macro:`ZFP_WITH_TIGHT_ERROR` determines the number of bits to write +per block in :ref:`fixed-accuracy mode `, the producer +and consumer of compressed streams must be compiled with the same setting +for streams to be portable in this compression mode. + +Testing +^^^^^^^ + +The |zfp| unit tests have been designed for the default rounding mode, +:c:macro:`ZFP_ROUND_NEVER`. These tests will in general fail when another +rounding mode is chosen. + +.. index:: + single: Subnormals +.. _subnormals: + +Subnormals +---------- + +Subnormal numbers (aka. denormals) are extremely small floating-point numbers +(on the order of 10\ :sup:`-308` for double precision) that have a special +IEEE 754 floating-point representation. Because such numbers are exceptions +that deviate from the usual floating-point representation, some hardware +architectures do not even allow them but rather replace such numbers +with zero whenever they occur. Such treatment of subnormals is commonly +referred to as a *denormals-are-zero* (DAZ) policy. And while some +architectures handle subnormals, they do so only in software or microcode +and at a substantial performance penalty. + +The default (lossy) |zfp| implementation might struggle with blocks composed +of all-subnormal numbers, as the numeric transformations involved in +compression and decompression might then cause values to overflow and invoke +undefined behavior (see +`Issue #119 `__). Although such +blocks are in practice reconstructed as all-subnormals, precision might be +completely lost, and the resulting decompressed values are undefined. + +One way to resolve this issue is to manually force all-subnormal blocks +to all-zeros (assuming the floating-point hardware did not already do this). +This denormals-are-zero policy is enforced when enabling +:c:macro:`ZFP_WITH_DAZ` at compile time. + +.. warning:: + :c:macro:`ZFP_WITH_DAZ` can mitigate difficulties with most but not all + subnormal numbers. A more general solution has been identified that will + become available in a future release. + +.. note:: + |zfp|'s :ref:`reversible-mode ` compression algorithm + handles subnormals correctly, without loss. + +Performance +^^^^^^^^^^^ + +There is a negligible compression performance penalty associated with +:c:macro:`ZFP_WITH_DAZ`. + +Execution Policy +^^^^^^^^^^^^^^^^ + +All execution policies support :c:macro:`ZFP_WITH_DAZ`. + +Portability +^^^^^^^^^^^ + +Because subnormals are modified before compression, the compressed stream +could in principle change when forcing blocks to be encoded as all-zeros. +While compressed streams with and without this setting may not match +bit-for-bit, the impact of :c:macro:`ZFP_WITH_DAZ` tends to be benign. +In particular, this setting has no impact on decompression. Thus, all +combinations of :c:macro:`ZFP_WITH_DAZ` between producer and consumer are +safe. + +Testing +^^^^^^^ + +:c:macro:`ZFP_WITH_DAZ` affects only extremely rare subnormal values that +do not partake in the vast majority of |zfp| unit tests. Tests are unlikely +to be impacted by enabling this setting. diff --git a/docs/source/defs.rst b/docs/source/defs.rst index b0210c1fe..4e29ac485 100644 --- a/docs/source/defs.rst +++ b/docs/source/defs.rst @@ -40,3 +40,4 @@ .. |carrrelease| replace:: 1.0.0 .. |cpprelease| replace:: 1.0.0 .. |verrelease| replace:: 1.0.0 +.. |roundingrelease| replace:: 1.0.0 diff --git a/docs/source/index.rst b/docs/source/index.rst index 0a423ea59..d979a4950 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -10,6 +10,7 @@ introduction license installation + configuration algorithm modes execution diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 4598024d2..56e6d4b69 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -198,10 +198,10 @@ Regardless of the settings below, |libzfp| will always be built. .. index:: single: Configuration -.. _config: +.. _settings: -Configuration +Build Options ------------- The behavior of |zfp| can be configured at compile time via a set of macros @@ -210,6 +210,9 @@ in the same manner that :ref:`build targets ` are specified, e.g., cmake -DZFP_WITH_OPENMP=OFF .. +Some of the settings that impact |zfp|'s behavior and what ultimately is +stored in the compressed stream are further discussed in greater detail in +the :ref:`config` section. .. c:macro:: ZFP_INT64 .. c:macro:: ZFP_INT64_SUFFIX @@ -252,32 +255,32 @@ in the same manner that :ref:`build targets ` are specified, e.g., CMake default: off. GNU make default: off and ignored. -.. _rounding: +.. _rounding-parameter: + .. c:macro:: ZFP_ROUNDING_MODE **Experimental feature**. By default, |zfp| coefficients are truncated, not rounded, which can result in biased errors (see FAQ :ref:`#30 `). To counter this, two rounding modes are - available: :code:`ZFP_ROUND_FIRST` (round during compression; analogous - to mid-tread quantization) and :code:`ZFP_ROUND_LAST` (round during + available: :c:macro:`ZFP_ROUND_FIRST` (round during compression; analogous + to mid-tread quantization) and :c:macro:`ZFP_ROUND_LAST` (round during decompression; analogous to mid-riser quantization). With - :code:`ZFP_ROUND_LAST`, the values returned on decompression are slightly + :c:macro:`ZFP_ROUND_LAST`, the values returned on decompression are slightly modified (and usually closer to the original values) without impacting the compressed data itself. This rounding mode works with all :ref:`compression modes `. - With :code:`ZFP_ROUND_FIRST`, the values are modified before compression, + With :c:macro:`ZFP_ROUND_FIRST`, the values are modified before compression, thus impacting the compressed stream. This rounding mode tends to be more effective at reducing bias, but is invoked only with :ref:`fixed-precision ` and :ref:`fixed-accuracy ` compression modes. Both of these rounding modes break the regression tests since they alter the compressed or decompressed representation, but they may be used with - libraries built with the default rounding mode, :code:`ZFP_ROUND_NEVER`, + libraries built with the default rounding mode, :c:macro:`ZFP_ROUND_NEVER`, and versions of |zfp| that do not support a rounding mode with no adverse - effects. - Note: :c:macro:`ZFP_ROUNDING_MODE` is currently supported only by the - :code:`serial` and :code:`omp` :ref:`execution policies `. - Default: :code:`ZFP_ROUND_NEVER`. + effects. For additional information, see the detailed :ref:`rounding` + section. + Default: :c:macro:`ZFP_ROUND_NEVER`. .. c:macro:: ZFP_WITH_TIGHT_ERROR @@ -288,8 +291,8 @@ in the same manner that :ref:`build targets ` are specified, e.g., to be satisfied using fewer bits of compressed data. As a result, when enabled, the observed maximum absolute error is closer to the tolerance and the compression ratio is increased. This feature requires the rounding mode - to be :code:`ZFP_ROUND_FIRST` or :code:`ZFP_ROUND_LAST` and is supported - only by the :code:`serial` and :code:`omp` + to be :c:macro:`ZFP_ROUND_FIRST` or :c:macro:`ZFP_ROUND_LAST` and is + supported only by the :code:`serial` and :code:`omp` :ref:`execution policies `. Default: undefined/off. @@ -340,7 +343,7 @@ in the same manner that :ref:`build targets ` are specified, e.g., Default: undefined/off. -.. _word-size: +.. _word-size-parameter: .. c:macro:: BIT_STREAM_WORD_TYPE diff --git a/docs/source/word-size.pdf b/docs/source/word-size.pdf new file mode 100644 index 0000000000000000000000000000000000000000..c34e1644ad4a26c6f2e4828464505841bc076c87 GIT binary patch literal 47962 zcmeFZ1yo&2moB<-f?IGWSa4?tcXxM(5Q4iC91@)1t^tC(OK{iV?(S}P=j5F9`TIY2 z^#4x3H@aWny9WcRR@Gc{)mL-Y{A#Y+kjo2;(lODq!I6(ncFn-C5;GE88@_|%=7wXC zHnB2ucuUN}LHzp%jzQG?y@QDzF@xxP0|ygf6C-P56F6R8IC}>>69Y>)mpPgZW@Scz zpb(UfJ(Lc*y?wxjHofPFwh&ajM+rYYKz|)t1J1BJ^m#6=N8Z~e+mB_CDXr&{b>9p_uF+|UIroS z_tthwHU>sN!IBqJBGzJ%H*f%%lNE!oiIcgJiK3XG4(Kj_T>S~Xfddm06EWw{sRH$9 zVqzu+{#FC^{M}A|>nUoOLJX4mz069?#PUa(jhN|&x&PLm zotTO3k1_`_6Z;=!PGY9t%|z4^NX+#6bAWU)5;OgL43ILAD&}9BLAscKsRk)y{-qnF zjro^ykUHjH+Clo5f2jv4Wd5Zeq>=g81AsI#|9k>o(C3$fGO@BV|HrQRIdLd_ksTr6 z`kK1wsN-e9v!#N=m`?n08wn*ddz8a~FWJ?@AqVM1%#TIZY_iew^;MG0!yk4m{ zdu8KcYy2#F_u)F!ck7nX_KkXW#$`_q zLm$uI=vYK4$;C85>n??;kvp}GPBdV>)1`8qJ#f>!s@Rq!kUl_Blrt**w5F&A=%o>t zyFi+mXoy#1D6VtFN^+@h>f?~fO)ZL7D&5kTk=xg$YJTT<$(WxO*UTs-mr+6(2)dzK zWEy`lr{*+B?lY)_hI?Q>3turwSS5+_Lqy;mvL<<1LB1mcO*Kq0S^`KSK5EWSmHL{| z`7Ux@Oi6gU{R}0S>5~L>g*|g1-skLe?A{HLPNY*o^wK_l`Ud;p=n^37^yY^_*VK}2 zY%A723+Bk#A~Y`j9rHzaHip=yw+I}+jV2W$kw9?I{D2Lw_h1UB4^pn%T(!n3izm}W z0g>38`g7 zM50(1vfkHmMdPyGZ|{!@R^(}L($d?G9}u9x$rxnFb#t)AI+bl|;^ds37_=z+$SKs| z7wia{KYE${iBJr8Bg>%x0>_c5elq*({n^KnWoD+srW+^+l} z+guI21*LXcsi1}%?DA}h*M4C_SrJPl{^8%7X(j86=@SCdOm+RjASR^<-ZskL`NzlS zRyN#Jg;9HU&`7y!>Uhi*ekPpHXc=E+&xq%$y1C$$G#`hK;W^NLBje*lpLMxP=_A7>DnM|E4K)7k>R^~-FHQZIr;2}-&>l321tAjV>gOo%$Fnv-j~LQ?gRZK5+p31!pK0Z(t|)VX85+!Pevrw_ z>*=XfA4|2tX^y?_*NK8?G%A5GM%8V~ZIlqm)GjW~Scx5*k|@fQKyH?2ARU>K7|5h* zXNud6-Ir*N0U_|Cs17P(8@)2iQ;=@<*gF)smA=$1xDpb>q1-NE@;I`UE=KDC-FUCx z4q=QjZ)u9Wf5g2qyH9J$i2X9JeO>iUvzLcE@Ts{atXP1*^;S{eT8fh~5N+lePdR@u z^N6)DOh$B;HGA%Zol}l~OKM1mX+P1-*KFH0PoO7osQmF^9<@gAxdr;eGGc6F81^;x zAx_-#H{vb<^+Z!^Ti*DxYpTdE`D%nFCVmpzQ$+H*vJS?)=Sf+6H&fvINkaBIo-*yO zAvAn-&c<;>jXH16NlA6=-BsW_2S5ACm41(*AeQnonEv8N2If`{zg0lNLD~Gh zxs}-;wV#X!bSF>{1I2Mx5S39^axk&^QG;X9_^rtL??}*(cq3!rU}p~EN+3$ZMoh;@ z&&kft_#-(EmRf_HTUVAIJT_qsG6_DIF6r=!cD!9>@-&Ivn(DtU%Co{FyiZn#3O@ z1!Mrf&)7dQUQuf+ho7v91w=sp;9GxU!as=cZ#?)1WBcn#{=*2ESbn3FzxD{^;pK4P zOen(lkD+uiq)dHD#uIRE`qaCf9V~7+C;mz&*KAnOrg<(9Mxzha(iYgY9XDz-V*|%2`=Xc*lg$8%3Q@ZwvDW@t)z*G=4j^yfY zhkNE4okr7#<~Gd(5NcQ;4p<%lDsW^9|7>Ev@x}iH)@=0bOh2{`|HW9d|Co>e9oF$( z2YD|q+7!n_0^IT;htY?9T1)F08CkmUu!D|DH|qnxDIqUv7UG%Kyz;%qIzW4m^a|S& zfmh#+SR>DeR%<`!8|#Wj6i2Og6$vTZredCg-Z-j^5aK*aLDV|;#{NQY=~dR!T%~59 z9D&cbyZj(eU!tS4SJi(3_7770H^63Rq-SIV@qA8tb~g6^da!~22)3*YydNV*W2@e2 z_+e(X-rTlh*$b}lrEVm96>3hHlCoC5BFJcdv|zz%gH$I4mwo=@yt0!eiGBQ*B^x+- z9a;mL`EFM1Zo2CYV@p_}PA;;s{K0_gxFPCo*DrUhr($(kQPmU}ni4WcRLD}&Bb#rZ z>rM~8A(pdw4FE9WxCl}p9>8uQ$zlEi?;rb)zkxUF532^@%|Xus{I6d~nE!Dh0Ywf# zN0tMlNe0TsunDZGkGPJ#y%G-#gwT()Lp}<A$cX`<>jKb5(@7O>BV2h ze*o__bf%w1p$8Ch5>x*L!hg=3{EW>1(o)UDM9;wnTCwRFSy=va2F=Pw%<^*=`eQ}@ zm9zMNuu^k!{)0XL6~6HHAUJ->fg6Q5+R$*s@C)_YLUn)_M_BNG0o@PQ@HdQ;g^8Y-1C+6$2Xg!;kz*oe`MG8P z+sOUnO7NG+q4Pi({W45+At-ndk}T*y472KoVb<4||1?YiLrF&YxD7 zfrXL$7tsAk6#Nb7nAt&~0f{;2nVJ8~u><~N$ofm{lz8mz|2N=qOTT#~LZX?iJ<+R3 zrZ>?mm`x?3G(FQP{*9l7z4PM$KbUb1SG)x*79eLH)AKJN{E=Sy8xVr5kByW4zX^mK z|FPr#EeJ=||G=>BCx(swnB{+g;XiLn{-wq5{|Gab{{zB5%n|Yu=Pl0vh42qg z7XSAUK4NiE}PsF6m(4zkWvA?+!VW(#Xvi$czjN@M*d;bbz zzXC(-Prx`w&~*L+u)n#n_#X(^uQlusz)sQh{sOSSxt;i*2iT1EA1?VjU_cEtlfMA$ zZ*C&~2LdMe?*V&tZM4R{&hTh;*}q_Jv26S!v~|qrq1m8wXPthxIf=uO91ppq!mGw9+E-aJ+Do z65pj9rq4)C$KkP*-8-ZfC%d_nZ8$0s4u8hcSdOW3Eya^-=RJ5DdPt1zX(@$DKN!9= z_>%kr=DwH^DejWz!|dyOWfQAe>xoOpd2XiDUSMrNQ^EKZETt!7^B02&h2<8j!Xwg< zLt;jhXcb4Gw&+nK_Z(UK+K?Du!g|AOZM$7xUa3gxCk?Q8lt@E_4P;V@wa zAY&v#MYS`61d*%jjtm{c;G%E8e0gX*q@Q!GYhw=YRy@O2S0sRQ5m$#8f!%IHJ#716 z)KM~GVpfa!CO5K#GBMc5gHiU;<%8{34|Vsg_d}G@!uK}xVLw<~OmM;j35vGpx>3p$ zm=d$LVLxet9@j{mD!2yd03pm5Q!h_azP|}S?`G-A>!Fk@Sl{T3 zYTNjBNCUOVejz0>od8DZzsi-92A2})7P#W?Ad2gjB?W0ifoe-?la=xyD5M<8aPbLv znn}|41<6bB0l^b(eD_Gvc0A-?@Zu8Ca zoa&JnSX*E71sAT8X472utsd7>J@fvx7k5XWw@WJJvfOPJ zg@PzctOhS-c&jPD+7VqRM0gWp^I)~}q3M3j{0ndS^YZe$w*Y^<%KVSsJ^t{P`8Oy{ z`)XxDfZcXBzE_I$2oO2O5WouvnUj-}@Xn1F!6rgA$hX%Kl@`gCQ9_@lZ0fnV@B^LG zy?BQ`TD`+r&P77?PV4yObW+%sYl7B}N*0~Bb@1hujSh^wb$Gm9^5uJ8W6Q0zxV|Oo zUcCU4^L<*JuaB7i8WAJtLC~Gz%e(eO{lvBJ)N|c+aLe748fZ%qbs+23g>FP(3*8-6HgaWj{%K-R*V-5hfPn#AiU{ZIiXSsrMZY>?&_)#6z#&nWux^k3$67@=(34(!+Sx9 ziSfRR9}(a-0|lssUTNzCV#4a+V-n20D=-1)1{_!V6UMl22~!AO!(lo zo5?`D^LPBtnxRWpnTNRb?m}k5VP;yXc}LKBXlkL z1hLEG3{hQjz@5a;SOOzlbX;MeMnpQLh^;#oPc?+Q9ba;TiWlwUn@|--#5JgGerLK_ zczJc2a0M}o@NzYlplAf;N);;acBW1DaO&30=9Ro@dX^g1S&0<$fT|^~r-|I%qA}$lXfWWC~CQSj-oG@sh`=x}Q zvwf96eVj_jU9(X0tK2-dTQMi-Suw3I74md1@Joujj}D=`wxrfo@)EiIsuH5H(51Gr zB2Htiuj@Pp_hkF^Gc=6aS*lMw{(UHYYmo*_Kk@-nN)#FIcSYF&rs1w)VBUV+Dz?Yo zGWR<9)GA(K#dGuU(>2BYa{TObOaId~`~7m`fw!L7Gfd(mk@A)I=3t~4^i>Xo=W8q(wS{wbNu?4Lv5;;+y* z^v|I$Ev!JAZTL#=pxR;}3}?o(VfecY54ZYR12yt?|EI6V5_2AtVfp$*PR~|ea{Wfj z(;xeew21;65^W-v_gD2LmWNrijB7=yDO!ZSYpLH z^H|KvRSis+g@M5N1XJ{~XmH7?>M6Ze>|5TlTa(=qpIo2>2WC#udzy*#n%v!*p`aIW zDWx+NRH`4$3!neriF}oRH}diQ-^ll;$QOTcckmy5zhqJ%^2-^ zN6oYeF#iDv- zVMVdD_gq}wJUbDEx?PLmK$bdGUcpnc)Y~pAK1Fn?O?!`k*P2jLIGVIZ7dX=P(+)bVN2r75U!Z9fpg@l38Ye4Ju?ExwhVbv9Dd_;!00e1H>I zs?SJY!jWo)!~l_-ECoWW3IZnlMI&!~>Z0aOtYAZ$F=_Z|JLly(RCgj#KKAzWiI}Xd z7K9upGHIsOL5&WPd z;vlTmrV!XCsN2{0>D;;DG3a98G03&07!LEJ4CFdS&?zW^$B}NYZ3Ople9^~o%y&ZQ zQQH!QJ44z+?9Sj36$DyFS++WZ0$*jnxp{HQav>p}-ze23K{>4SA{~$1(@J^AKHHR+ z0NI1p15N~h#GcO8!! zi3tZW!nO09ZC*LI)$^RsBQf83<=$Mnjwg=941=0h&tvV<^OZqlaeSLPRp)r{@obTS zz#4sNY`;h?t#Oz)_^8|wnINsMfob~sW-_g;-*Z8T_sLTKLJ`s%%n+}j=P#V)pEp&Y zqYdmp&}sXhuER{s$p!>Pnx8dLqVkU-F_4{u{XeFYVYN`o$@7?WpRlN{a@EV$P!<*! zo8Q8sz`a1!)ecqCmK*~QP^6NzrNhXtld`3z?{eJ*Zl~EFJw7un;o*?=X}i~;5jgTjF7HQ(7SI01n16BA*f^nM)b%t5xb2Y+t%88U_qY!uX& zAM;k&SkbrO@RwPdLu{Y)5dbSZk_8a*0wOY~A|k0!U{|2PMV^HG+eN_6z7wKo`9Ndw zzl4TrG-T;(*KS%xafjg$&U*HMD)UhX<0d5)#P_a)FivT=f$hbG8!dCP2XK)w`+iLY zxG|zY9&A18k*IePEG%dX*Vps#@YH`z)v4FFahFg<@}cY;0?UARg8IhOR^ziQ1HoFH z+Wx#72A}*Q;}~H3XfJ|ny1iwN2m^rvO$iH&;3#raCuG@f10dz$mlGk#*aQ!Jl2Cn) zX#=~la0YN|b9)Ef58OKw!(JepS)jmf9T*~B2J#<*IiXN8@=GbC4h#*1LFgk^Kgv`& z*#(e16!Ux~k^RmN+d)*V1%YAW0Qeq^`TTD5C1g-tUAWge*Bh)UL1z|M;(@cfF=i+qTXi_!GyBs z@>#z{060IfJoYM-*9(CA2mCtk{LyPCu;=pL^Z8FZ9V0zZTi|6vtl(FEGapciW_U}! z6YBfmEf5@RpC1Q2i_UO!1GHd!5%nQzg80jyL&6#T4xc(>1Xj*T?7c?uaM&=zTdr%PSPpJ3y82}W2m-Sok zxCbKoXK%@8g3V|8t7kfivBkb!WbgsKPtVBhb2w`ok62J7dd?i3RBzi*Aiy5=3Q=!H zz`g_K1s77EomIMqaGB7U<=y?i0$p96Sp~IUlIZGdSq9_vuF6e4m~-6l*vg_%spD41MA%0LOr=eB`le0`~=2a4yW@;n_~Yz5<-A z3Jm7b&FmvHLDbb}Fmm|A1I99l+K`}oo;*unZEduk*&jpv!4}w`iBSO@JJuJCZ>;9M z1qcBgH`c-8!;jlr4?-&tHuKMjw@`ouq$PT!;N(pr=nm*xEMa5kJ1pTG=O-A_*r7)T zy-r5kZ=j_TY~jRPK-hTN+m0;B!Lh>0@dgV_;h3+Qbh7Vo3jp(6L?q14G2%_h-)ZxV zC5*9s3-eIA4ygaO8a%%nQ7(`&<$Xe=MuL3+1#K`Q+s<#88LpB!6#tSrHZ2A3+uBah zZhUuB$~P#?1gk|jqc}da%L?Z0Nl6T|)-x(i&KmRnmZm}7B;HGn(!Ao+t=5$sm4%x8 z>X3A|I{q~Avs)1nz5u<5ShLvIQe>Kt$D-qT;S_IsqARM8zZ-R2o0RM&Nt}6zDt@KP zxogmESfb@8Q~DIxHjBQMqOx5VS)H&QWuAm{@oueHZ6+jb6Ov~6 z&K(Q3Rnb`(roau-8lNe!9`$wbrDESX-)jzSmk8%%rH%4roi1N$i_L?)ongsE15eRy#ids5sOAR(nd?!S7E^=Ukv`|jr@Kes$sN8?lB+|9% zHC&NH9H|)z!V#LX%Rj7QTRxx2FSZsH7OB|xc(2{UnxwT~_tI3HgH?TVbVS+6DzSJL zu3A4+ayVm=DZf^*=1U7?G5F9o^max&-xzi8@FBu&rsc8=HBUo^o^$asCRIGXrfdkJ z;^#=UcN@-@7G=~B8!P1ZjDAZ69gq8?8rhPS90&-?L4H)3RrrctK0)USYvDNoL6q|B zwh6t9BfchGvH7kvkw%XzvJ#)7DR7csHP5)2NuT3*Ym-raBDB>;y}pMwHMzM^;_M=H zgDVi&6Fyl0FLLfm@uk<|Ex9x>NRIJQcN_v&V)vHv73c6X#Bi-cie%NF zy+t8|nQmg9c2eA`Bf4E?SZC#MR_xE@DwKc?u|EsNyAu3%Se5o_fG~u9`n8ig;UbG9 zJzD;@UlX^MNZ5Tgq7dS&3+K0R258FxipQ%83+**Td1w`OpZ=6n!>{B;o!0IHWrZCr zh;^zh`hAA?d%id{Svu9$0k?yx)2BhodUulBY9Ajpl6{jVM%V0TFKN{?CNas?TFD(% z#(PC6)-`YIcI?@#-h7xy4NLim5|lpJc}R$Mbj+IpvA^H%DYV}c5++f92R|Oy*9T6m zZ-5z{bkg%xuk>QDnw#1>biz_^pJ~|lT&G&4L^1dvlse#|33^?xSw|fQ5G=${lh+*%SVZ$~hr2I%jZJ@gfY1I*|C7q}quGVr~`_6vhBSgnXW4VVjKH zDy^a;#UddY4g9G+%Q$FFc2e+bb_yCM4YXq7my);4-d=@yk0PoBUmJBjj&3Xw+Hh|rZm@aK$9ot_e ze(Pg23#y=b6jWsl+_hKnU+r+MpNERPl!>k>dXc%R*Y|=(n2AM015Qjpp&1hVgQDB= zUZ@rG(n>>aHH3KhDP~P2zKrC2#XQ$FT3NZueXjyppXH$m?6fs|HIbo;ZfWs(%&B{+ zw6HT~7K0DhJX#=p{FG#dG5S1`r05y$V9;LPIs}-79PJT(3srAy1qQ+^mHmorhQ<+k`FC((Jw+-|a7C~m#%w}k&{<`!yDP^qZ*ZN`(nOxuI0kla6Q{2<<_ZG34qQnS%6urqZ8!klC}~v-6s=k zRsM+!J3&<*_f5LY(L5|J$6=Xrg@~Gj#yQ{$;k9CCO8r(W94>XCKhD>6))(>sb!1VyO}R~u^Lo^KEqf4Mfh6^Df0$dAs#T*c#;J0?c>QZMF~ zK6&dnYh3){BSKrj8dc|N04GXbg*7Ko68nzt#elS6g2) zwji});YXOJDbi}FgSVtrS$j6iDrtXz?p-f>>`53ycy6vneW3Bczk)3*Y(js>ndS%{ zToP)P^y0BWTPFT6j2x=HI_{{jR5*F+?Nb*0p(r@3SaRwu51w9>>&>Vz0|E_iZu#bL zKjmWP=61iO8wymxT+U}}Q{{8)`0vKC0H_OTsz@ix;S7sAnF@5a``my>rQ_(P(sE3D z81sN)zHR}6Ply(|{U`xdXI3BebHoO z?Y!u46oqbRb^x;(d?a^vU&+3UxygJ`8Jy26dPsR{7u}FT!eG~zV@GbxYUbX-((_^P zIQFb~JGoUE9fSH>s=04k_AIvNZ6LVT45U&1F`Or+Cz(~N$B29{_C4>~=3x)jkvhAw z?3QZ=%9l%c`>4a|_d*&^l4@f7%k0=ZV>q611+DvCbD<&h&ZC-9moXf>+%Ga7H{U-7 z*^(8=!s}o-wU|hC=x~SX?#z%M#=lM@&b3V6RB#NR?mR18z=Lr}XThj^2tg;)&;H^V z-i^5>dYE>G;gBdE9*EqDbR!JR;h{J5Shy4|Lbehw^KA87k@h>YV*xw8}L%i_K)?dC4pX}MII!N$P;G7_*A7k4J zc<*q9Sp(Di-tbIaoXaeeTdh@_vNxrg5QMKA{?E&hp@ zMTzA6qgPze5NZ6fbFNlw5g;yI=#o8)~3w%ZfL{RPNWvTqvx6i z*_<;rMiK%&!rMgNj%H<#&@4|%q^#h?`9brY|FdFPnuA<@2_1`JHU1dF$(4-OAPMp% zwIVkGqKwHf*}QuFsOe}P@cGH@#I3@D=`9wECT#J#tg76~PNOrWL1u@~E8zKtf7X1( zoQeElEN~!UH+~?|;o&R0B+QV-eIUP^Ahao>;<QizVSEdaMimYQZ^o$q*(Qq&w?T9W9A)271n2LM>|>+2PN&1)h7uf9*mrzr$uoFtDvdR^Qlp#yK{&Z@v-Iz*%fa6%YQ(HY>M zsQIm8s$3%QWx?lninXNwnTgMoMGv>n0Y3F!sdO?S?1p4RCe zk}SfjXYq-o8a*+PVpyv1^|wr0o9?Bt%qy~dt3FvQ^$Q32VAt6xY2A9 zS)eXL$6mT-P9f!a>71-ev9!-S&w}vw+Mwh|+k2k81*X;ZMIl|VW^7+Zch_4f1wv|6 zyr!-U%-57dGhlZCwK%-UcI+SXq=WL)8MIVszMtbZntf_^2n??4{+M)n8CFJ0@X!Igmy5=4q{{pF(Y zWC}Co9|yw5Q>q0?OzdUQ;B@eKB?H|6#j-q0BaW?9SL+eBhPK`zJw{AvncI71QdRO-cVJB31}90$*x{E16+LDC-uF;zdIx9#1i-} z(%Z<7APNg{IQal}xWz-)1UnnQ?*ay4_~1k5=OLn+Mwy^ckI0J>|#m;D-r&SQ~2T%&JY_G)`;tRSCTh+!ba_?FG!OWdz<}b@F`SD@UG9lZm(1_8q4H zmdAOd-(1FiFpD@l|Crfwn)j|bo;-ia)F-!)A`xz`E0zB;*(?xs+$Q~TJ(B!MZ+;;0 zAq;veUE`#SImcjIJBY8Xw_Ut7Tav6?EQR$y_^YVPTh-#a85@yQ8n{y8K&6e>}S5K)C zb#Om3<6~{;Jh1&Sjm+wXP)4CJu#jiTS9mIgzsQEaV^Gt zz5KZ%Rty%BH zWK}cv<|(gIaoO8ydB@E&D^wve&|Q-A7KklkK+6jZP-gGXe^r3#T5F=!fYR12z|t3B zOTfNYyu}muI=8r;OnieVmyL2%yweQsfNr2_94)v=YR*BYmy9N$+$1a-Hr(2;a%{Wh!lp>1rxE&HH4@jKXOsbC#ui1NwDAStGAhf zd(l~zd)AZr5zPe7TqRI6aju`aCA^XF4aIT|iS+*Gja+8O8D%D(1Bq46CKx2<;}8j^^@;HRlO;6LEK`PuI;M*D8dF zO=YBFdi*xL>wKGh6yD%6Fq(@5!UE9 z4E{Dpq(h$=qdo{mfJnu3dfr$4;}S~C!I&8EDzR-$gD%61jGJEzg zLA!Y9hb%ha5d*WG{2-KVtfgA9m~IU!peJ`D^&%`>mGivKyg_+s8!Q>87r@F$oAR)A z!2Bj<{3M)~cZIa;^KBHbS^L>G3j_T!s&S?o7d#HSuBpr`NGI#=_bspCqrvB2Z}i^+ zO(=o>uF=|!Z=qkRvVt>ehsxe{Qg(DJBu#8zVXQNdXk1{le-vCkvdPCOZ9wKt6AJhK zf?C#YR_aoLnGsg){spHl842O5Zg#Dmu*SjJN8wmHcsXw5?;Z{U0<2qad*4Qu#b$k} zjvq{12Ja-3TJPQ2?J~Q|AgAgPOzkX`{KTr5(G3wUlA2%sTOlw%8kvr=uZ8~Os%}evFX^tk7Fj@u2V<)YQ00L&k;iZ*Gb@EX0 zUA4&ZV!r$8DpLdrt+RV^{~zK6XRpEX!s&aG&gEGDc% z3=fV}Z5LuSK+p9BeOeH~u_p(nzwk3e_+7ZQ6{amOu# zra&yVRX_B#moVWBTWda9xio<|bhC4mMOuWS$TP}d-U(L7_Jun^N2pl#X<>O;-8!4a z)xZt*expiImnF@d4tIOn#!jqU#3I;o6_megRM2D*oDYd6ouQ~MLHow+S*Qbi}v z&=;O?>R!DRN@D1g@I*8y$sP-B84^Jso0E&TmDG}J4TWm8hO(^+bkRaXw?3+TKCEFW zu5Np`(_!?XNhVeDDw>csiTr@0V7%Sa`&Fe~prCP*OzO#Or}VyiJ50gQM=((jS|?dB z+IE~pJR*dr!G%4~X;rY$2J1bHq=6;%43x!!)$1T?O}gyZVQZs=O&u1E2ryowaWW?A z3WX6r=eI=2mVH34B7x7@x8=gid-FVU1Yc7>m0$O3&T2Gxh5O$(nhP8Rh^Z;)7dUUr z=bX)prCSsee5mDHr55vTt~y@u^{PT!%g9$rD_@sUxXwACEJ$(_!eK%Sm8FOJPK3Uu zwapyjS!Th9GPOvdn)q1J?gUhFB6U-Kz~P)}pbekTm=aZ{>+z#2ny2R)Rdn0Ef49!^ z!E4!}u=ilIdN_L24A<(#DT1PLv7;*USOzSYH1}i$6SA>>gjluoE)c!ySo!`v+@b2( z3hNiRhi`qdzIrO3)Jr!Zy7ICd;Py$AG|k@4DjL6pfYFn?Hp{!kMi1!?em$#OJKjKU zxX{4f-tH(}3a2e+(h=r1Vn*1fQ+U0_@%^Oh)`VO#;Va%5YP^^VQ> z!mC2>Cm)tb<#io_G){T;(AUo1y*?@Gs`7=69^Nu|9h{0APcAY-P+iq&g+SEX)GGxOW~V4~T~+_`~~Xej0h*^M}? zc{<5XFFY^=Qw-SjG)2~D--et9()9B&?b|@&un89^#e7|p4_DhcJhAen0kS!M`__qt z5JPdex2j5F?^fv6*f!fygx6iW>^d!&iot;KDnPvHl5=FWty!5=@cs8B7r<6Q$N9Ig z3IqDR;FUbp&v0s_(F99n4;&5O2}X-2zMQKTa~iyA*Gj@tf3qojQ$TV>MwYLn6Dwezf7oZT}qX&cD$|EAa&6ZtP zs*hH&@(X$K2ybvBAroGHVIXpu%%kDkx5adh!f|m64Tj{^Fvu_TXzN}{PpWF6uuh!bPPhan`^A{yNh=aq&-8H9_$@yXGcWoUUdkH= zn4Fr!n1T}Lw?`)d;D^rwtZ#MvV*bIu9uORw$4zqS=20vjS98-}3b)s@P0gl^p)*ruLB%h=o_8~f-9;4XU|~w*SbY7pAsPO4gkYlD znQ?q%@D}$M*a@Dh+-cp1;L#VPfz6D{6@v3L@f(sy4sR`WHYp5Dmv6=I6mU5Q9oo1> zU)tiSYzul;Q1tHTybkbE+Z!DX9xCT82J}Z9vlck+w>PEJ2uW&0(5+B7NMhj=tlJq1 zFcqH5oP2hC$x#~ne!FDs-SCr(z0-Z5FAb-LlDwWDvoQVCyTpot5|&}i7U{YVPN>eo z?iaFtXcFk~ta`(-_B`~nH(h4n9a?fDWJ@JeWv?;OGB_py_`7$aTec7pe&Gb-0<^K@ zBnbL%v-VIT!c!)NZuKkW1=!qO%={xyJ7f@sxj!gSvP34Rtgq}b?rE&)mq)kZ+9$Z^ zm_M#P&Kw=q(pHWnr%p^37muLt8ZmT$S6y@8eSAD9PS5FObnCU!vgZeeibRy)>9x|p z0sPh^mo8=+pC!$oE{-7JFHwdR{2ihyu%CDyohUQ=T^1KS+^QbVlcywT0g8{68leVe z_eBj~tlf{b3%d~?5(`j9Uf!Y{N#xPst#%p20&ra~b?o*x?)X?h zrLn6aZF|5OsGz%$hx5%`ZTKhd(N;EVolYMyXiA4oUf=mfMttti{jM%l-Z8nep{n|d zv|N1!d|4IVV&n1Ht~Is{sPf_~opOi8mX6|FO(I?Mngt5tJo)40Y`HA_hbXuF)N1OZw zD3_lD=c^|lJ{ggQVA}>|x5ZAt!mUzS8Qqbet{wV=1z-LQ4%L{%hc<_AJ$ei;1#`nVuO-j1n7lHtM^K?21l` zK7_Ob3e1Kc!3F~rt2p=8ei=-X+#J)MHd}>F3%=Y4T>KBo-yn{U{a?gJ9W55M+F(VN zYF8Yic<@5O0mB>V(u0!&jy*Ocjnv#;KDlzP1&$I=X&P!zM%{4PI?Cz1vdq$VioUk7 zXty{Uw8dLqr5g|6NLnYS;k(3h1&{qwl6=ZO1-IN}V!~Ov>GyQ!DqmzekDa@?5PP6w zYbB&IzmEoSozYM2BOV&>R!| z3`)F_UKi^r!$Hsx^jc?OWhG#}ZIWSM;bksuzs|N%X}qjb)+kR(vcWi_L?~E}N-$Lp z7~LQukSW@ZMS#Wc-r58@BQ>mY3~{WUe-&Adg$!!SiE`uXKc!)xqK-FU#TXJzT0Ry1 zZhT%N$WtI2t8WdLZtt1J z%QYf5Jb7f(Y%vQ|vGI$hM0@KBS;t6=-{UrHb0s#eDXDJoATeAS~4}-rc7JA zi#7YWpS9h`k?w}!L@(I%>IoT@@Z|b04v+o$OL9LCh=C4|v4Flk`|I!+=*}q!%+GHKgj2i3BqMaADb3GAokW8pn`z#>D&Bd8z;9cRsiHX zS7M)zEPjI*I0h0pD6mxE;1bX9!5zJTskSs5DtR@OF9CubWdL0jj5!zxXS?pzfrovp zUYL3)jJUYCjpvt0%1(X;D=3U;U*Nle-xf|@lITN@VncoSX2bg|-cGsw>073WzkgFx z6J)!Ux&TJE#40A=D15j*G;Vtt<0nc4xCbDkJn{kbJx?OB57yf?gr27wyli}1U3~Jx4QiSVVgge>^H`xK&Gd`FHL`-8~pdV88aC-{L6%)=f&g0dPrr0*%j z6Am(bd)pZ2S7OZY8(`Mx22$S zX1L9($}tB{10v{2XM}WGa8!?tS`X}xMMfihkSq6O z;aY9pTjN5j8t@Qjk1}=^k({u;-Y=e83!Z2;yp3)~XrHnAo-0Wlks;UOlaJ${z3bYo zZ2U)v0BKc(oJ{}+1s5FRXY_YS_jH;%*ceBvSLwPk_}C1PCOPP7gWw&8zQpCrPMoR) zM~Hs7ZMG=e%dxkX9FNp&koYfH_zhP1jxjwjU@wSxn_7rVZtfvIZf`PRAuyzSe{c5l z&&c3&q#&gSgEV5)Cj!9AQKH(~W~4LfM#?8#Gf+STKt@sF^FjuvGXZvjI{;pUMU3*W zVSI2H7^MaeJ2!}3jbP7IV5e>RC&PNuZPU-wckv|#jMmtYC#wW-r1zt_PX_b0#mq(V zZSkuEmFEPgr*WREYjt=^^#N0*WkjM1?J`$prZbE(7buy7cm2`>?Sga=JQLe?3BfF0SuGAE|v(qs;a#3+%EN8_k_1e;kqay?M4yGb6JUU z9+<_3EYN)3UY9o}dPzIFElzS_!YbGV=WKd(B?p-g=(aHirwJ+9{ov6LE>`1GAy<@+-w@+FPj}Zyuzx_7{pQ z2_6?AlX4D!TUVa!;Z?Z}-eT=w0K4yo9TI;{1Ids!1+2yO{`hgyTnR1AfsC>nUd)em zxM|KpDan#!q0r}9j%Ph8rCVU4v9IUTnml*Y%K0A#h8}S4tMs z$-g!fhDrw*jl__zy|S<1wVr+;QMbAr6t!767h4yry}wPh+x9NIMu!>`mtFA2)=rec zupPWhp0N4FlllKlMhB@*v;P3Dygku$>b}WUa;;bL6PkUtI_W$*M`g`e9@&dO_ReKW z(nIg#XfU}wGd!$7Au}<1#ajU-N^QEZ8#AwGdmfOkvVjb0E9xIE<=zzi(NW}pBB-?j z9;E`e+wU6IEw(#9xcz~hR_hefSVomq3QEQ0jh$PojA5hA=>>1JeG!2sXe!rDv)8qh z4rT2RtTnr3;jZC7PBbq3{m#9EeYpmeE*1yz%bz2K)inyJns7l|Hq|?TB3sQ5aw^Ly z2E?S!f6o>{042`gLW^4V=EDfTO@OFkI#&Q&Fp;n%0e!jzUyPelsT$#}Sfw<$4YYwg z@X$N$^`z&2qGEm)oL9co)zgzr{VB^A(7-Il;x|)~beI`1tQv7CS~2+=L1I(Ak;pMw3h~K4W4!%IOjA`z5YXW*wKlnA9dJ}W zkP%5NKotsefH-9NFvi19GofTn9EGe@mGEg-Qtkh@xV_C9CHfqauc7uUsHdi?S6tgP`Rp=G@J zP7MZ>$rzK*mIKyA}I6cEqlOl*C5#O1uYtPP?P=Qb&UpL1kLD4lv_2ts^ zg}O}P;cHdCo=kJxJtgPJ`x_v=BKBv67hbi0x?A-(VMe6J9_F=r=Be+3B{5uZ^v|CjZ z_ic_zHa0Rp{!kuj8wFe(CI>FDglS_4F0m^ieb@ zt*>->_C7)+@DB!cO;3IbM0!ev^eq5F4*p|*qA9Tj{74cr8}^+S~V?PUr|)`rq7Xl znf9_dvx*>pL&OW7hdozQ=4hC?e{BZ=^XRC{5C8jJT0$j)avtX$5SvPja3L`0oebd| z_X(>MEScqsBNjkGJNOoTeaX^(nWze_M5A(TP}Ak*l>Ta2B8x5&fd^;hs`n*SB}$A< z)T0@C-|an~IkEdsakWY(5X7ZF@TnEHnHif{ymnOLJ}PFyW)>gFZ1#xf%=n8IR%0$) zQ4cjal}W@Pwdl?#RA}iWAoBST%Mu&+JGj&0{HB&it%4JWc@B6z^w=Wt%Wj)>vHTqM zjcUZ{qh}n7aC!%+5Mf1Uc)XlHxApP z1%N#k(UOcVxZ}-K5Rvg$WZbybpNY=?ol9B%8bEyPHp{&oy%?;FR+lvWM;onBZK0xT zWi<&PB}KkCiFb&XFAQcf+&Wg!1=^Ye1i5Br#QiWmtH?oQN$GdF4Rz z)#cVT#CgFkHtAj_JY@H%AK1dhpY~?_*0Ds0G~_N@pBdz(PZMdu<(p8|vZK9> zGP4sz?gyjiZ$;=HuOCstk%E41s#nU_Y!{gmaUo0Bi(y6Q(!c{f?e%e=KJ<`-bRKo2 z77zS=al=371CQ13X%YXx-$OAZ0nMU*jtTput}_}NO2)oOI5HPh4!E1TD0@!^ChM?q z!}WAA(&ey2<^f!2Vd@X!9!#&Xzk!G(T>S_>7Lsr}-E=*DFGaG~9===Gf)TG|CtD}i z4-y$d)U_K(SQzt;?Pv88rw3O-TzXO#DdzyjYRDAyb!fTU;ewqVaaHBs@mVjMYK{?( zU62iw++aejX~Y^HF0g6o?|k0WjT6k2@H}H_k)#kO0UfSb&OyWb7cO%JS=VDN-W|wo312)o#G+8+9+3{LPi+xJstY7jm#s}4V|sjU`HA$`KPTKUsXD}?d6oX%^Af{` zs)9zJ4?l8#-7Tq+`;%<@-8q?R-};MTjhxk+{`lIo zhs)e_Rkv-=AK2Cu8~2Q*d=m5)kmqU=sXB*N#O08euO=nR1X}XDK@66C^RQUYyH@&(^ zT?}ly71T6NO(k1|lqc9%*H}u%_1FCe-O#b*916p+Hq8G$_!+n#V}*kl-w-t;7z|44 z&z{o|Ilv_m_I4g2LloU!2%naHbz8n8+^CfIG?J=FcYtzkhqBh|XM}gMVxgwZ&EO+( z_GO|exmBE{=L|jQx%C9MX<(-E5RH7)r0Ma9YBK8jQUF6wmfNZYnK_)760dVrCHZ60 ziL!N6M%xh}MFX*=B!Kg}e}fChkWwn4Wu!$+{Q|i5RdEF**ih@>WvL+%j^J+zJ4;E} zRfqQFB~;Uy|%gRSOO_wR&3mSVyNhZ#Fbor<&h66OjdH$=Q~4AJ8n zjYGg_PbsZ>14D{jti+iUvo4t(BAdNw(mq2ATfwY2B1R1WTSAvjUeAFPiiJ7(Q`~od zLCRwOJZUgzq5~KHspMm|N8y0j0{AdM2-gu0HGqWzmf7ONNXvMC0>UAe-DzUXrb;%O zinhyaJjIz5w@~=^Qr1wTBANheybWRew%~I!B60@3!v5eE7@lY*8!z&=b*)n2n*?6cqFTvc1v5##qpt-?JsoiVd;>JMYaNLgAY{DxawPOKPvgw-Mg`#rkS zhX#jlk^AS=a;wyn|C0tF_lJ?eaQQ&K!vd+eCD@|`p$Xo_>9h7_@Wf@BVDO86jSkGC zX~l|0`h!WVrzP7fMZ?yh?vY`QE;n8wi%IU$`xR44sRDUEadiv2Z@a=kw#OdHHOcKN zuI}|sEDu4Z&8_6%ds1(LYzx5`|RO~)&B{a(wF#DY+rzY5~{xvPV^r?_jHc~pJt z>I*b|V2=IBQ}sie-=!v8>A=ChE<rCH z639@LbAZxUK>E{sW*(-P^-RL;`2@j*3xf}+Ah`i|kRF+LY`&L6dZJ%>yH?1StiGU2 z=MQENQ}~|z44svf**4$6eq=V}4KdDj>Mj1P9nsZ(LZO61W(M{Ga<~2`=dBTwo0PD` zgAETVQ%{Q$K0!)&c-KP-$5#5?w@tT%Q{?WAy=f^+YdVBsz=n!w{2A7j3&EuDhF z`cR#4ik3NHr~>UpJt;P7VVK!EFPUl~Y8njm9rm`UIC74B!@QKub$R0xl{EO5M25l3 zc8_BlnBgDoUDC7|*S9d4o+!8;M{zD-DIottgx8!mP%>R&*PCkxZGY+-e2K6-FmiA) z3@ww}7QeoY7>x$9vXB<)TxQ{{41)8k);iv+2h0F2Q|U93))i`HZMy;pN9n~rrn~|u zwjuN*=P-N7J=k!5yGHgOC>EW^x@L70{+>vXEf;8BcOq9fyI3&gHt7lk@Vlt7dY?2t z@$QF~w`BOwhSE{6?@&j{vnW1|Sd1>@JHhY3261y!+4C?9prpzhKK+APL#e#f3MD44 zZdUvW<&C=r_anUU{aaTThEycpx=_eOTq{(o8E*a@Lw_7-a>bWivLl4`Wn#^Olw&*7 zeaFd-J#rpKQq0{VnW$WDBWh5TP%D7v@7RG9Fa6UDu&ebrJV};OzT5>L1P=aUHJylt z`!!n)Yr{Ad{@oO0J)kq0Jh&@|Wu%kZx??qb#?_9P)4XcPmk;j}K25kB#UcsCJ5R$( z>9b3>R~iJ9>rs?8m1;R^6jei*h6`k?P#0tqBa=7jn*#mgQP`4G2BBPJX6%$4?(C{y z6k6$iB{UHD9wXy47h;|)aigq{aeQYZuYNR=LjAAj$9Mvh>+kRkh*T)KQ$kqhqLi5O z!@YVhQy>LXlDA>6mj?c;FVgYfFJfMdUy=wjxidof7sY51>OY-D?U z7>ctP!;9s4%aKo`qM(}__b}K+o~-vCBif_G5)G+z=CfZM1uq`BXoITCMnhB-YW!7l{2L#j4-XKs6_0sr>0c!2f52Tp;op;$Su-SEraDD@8xod1 zAN0H7<{B^cW@FSGQlbPvRZ)z_DTO*3lpC*tXut?Dhed)D-N$%ujXdq^iWzWyY0FE@ zTi0-z_N06ji50ys8XbiO7tXS165>q~sH($W%(Q}gix&{aYWCKbXQ=nYe`DV)*jh%~ zya*5&2)Z>7aoNwFDBu zEBaPSxB(~dQNl=i9X9;fKXgUK(k3gJ;<~!pfM-Xr=tRassasO5n3or?Wx|*2q?ZZY z`$uD=SKErT<=#0O=X9YfFe@&#*GifKvXYofLrc!gF=oyrIXfj1jnfT|4erGJvpn-i z5zmTM(RDO?;a@iVZZsGsCG!r%;ebV+_Sg>O1A-f8BFn56q8cg% zBhGUjkKaOzsv(9IGpZ{%hsk%k0n@M-*AdZjY==31flWjbF@PM2fX6SqX5~%~ySZqI z*G;*1?P%I&OLhxBII14#hF8Hxlh53cfMzBWF2KiA(4k^>a&=ZyN{HFc6bgwJ-)Qu@ zvK#-0Z1&$#iT{z!SXlo{He+L>|5rZvzcUxu*_i*m|3783Rxsu)9W+*Mf`EA`Ck9Rh z(Ocng#sVbaY5g<>Q}n`Me90v8d9HcFd99=~8qWDbQ8a`C5xQLNWAB{T@89Mg+ilBf zZ#_>v&)n|W_7@GywRGpFcVsJhReBU;#6hw#I(!htg{9R{@c>}ppmz}9Ait|CQ1B?C zD_gF(ZTJuuk-Rv?UqkYI1No`wYXW3Pp(SAgfeV`$d;x>FytJgewj@6QadCeB2U|i& z8W0a~XR$Ru=68V$a^fkJv6>)<7eS({y|hf{pC4d*4aY!W5|Ux}UvMxA_Wn6=OLD{_ zh~Yj1SvD$7Asm4Za!X+7!)HB;Hsd|?s77S;tNs0cIA@1rvHa_!Hg^GBaA@ozFvsBh zZ3EW4U7EnAJlVQl%|oz;K7S}m*!ENk>H#|39<84*@E~aLj#nNodo7GVSg`XmocO5M%o@a39)@&!GE$( zOf!??StZHI$j}iFA-{BXg@7?ly&G?J_j+Mh0mC2oJ$|FD5X7-IcdggjI0EsA5NK`a z#|C@Z*+}>JSh>(ai7EfJk=i?A)1$#pEn>p{bo=?IGF`!y?%6)dz|D4NE6m_;X0qzf zBDB@Cg+e(PSCN|3VrR|IO!{<|mG%$ZjHxlizP0yy3 z7v7N$a`ey3qVF41LplYzfAZ9;3l8eR1OqAFi6!y~0r_n%+*X0lZNSP29>ww{H~wYr z`{lWYjUb)|JArcl>Cp~>0RM)!I~}CKwWUo^!`AUN2-1m>_p7UnVIJM^&SNfEV2DqS z3G)oJgZ%`92;vU46IBE=^$152*!Lxld_oHBw7BOOM*|7-3y?1MAlwpw?xW)$$>(ZkI4H(A%DWDG%q69J2ItUQsQJ_ z$^X>?MX5P;3;{23f{lx`ay4;PtaaqDD;7hYRN?|)T{Jx6wFq)f2LNY8t*H79&g!b; zXFwMqn7<{LM%MPRVxJsE*V_d@#NF3K^#j(Ev52 zD0$*RyyP7j{Zc`+V{LntNo-|&?d2l4!?SOLc2f#Ey}=1?PzYg*aj+fxAhHVRwOZN1 zmP!TkohGxUfUZIT4MfHqOXQ4vTZ;3$GH!^?DLf^QA2SA1%-UhSp5}-yA&03sW;2~p zSnjZAFwKtd@FKwt3o4uusdtma8TNOI=K|Z8o7Il5N0CclynwUm9>jxh1jC*8PGy_XEp zq#q>;W#ZR1p!E2+S-r(L*z|KFe z?-dO)ZLHf)I5Li8&6C90N#s@cv?lEDi!7dA3cc|+>ni&8YyYc#1jJ_S*$0xy@#yW- z#Mw5N)rIRthDqkGLrpErUcd}Ho1>w(&(LWX(7=Ru@<%}A0nSRcUvd!Fv9fZy#DiyQ zH6(;`ZZ;aP(mCYFE~Sz+x46y*U(FM;{$;t#KOWtL8sZ)HaCxbIDt=>^>6#eI<~*){ z|HS!G8f8?y*q%0fOV7HStMR0*y4& zL?S`MiDQ^kV+Mw%6;=(|f#7UX&L@*XXglUh^zcwXr6fiD;5+{6JaOt?!0`8Eou zKDs7VH!MIUFUK52A{?zeS1>15pi&3{*K8bYMQwd`P%-9xhT8>2uZbM!>#Mm=Nf)R$ zR#>7--nxLvNW1W8a{{7$TO+jCwj!*Wb(P19JwLj%vu#&k$)k8E-Gf5t7-<$IrLxtR zBA5qnyk?T#=BYgk-&bm4Vc_nRqttOBdnk5PDf#-vad!jxd+u9CwAr1~K`a-H&vwTk zw&yC31YnFf9ePh2xI4NgnL?C$0;ZH1-c2Npr$2-D)Yphe`2s<6z!BTNH!NUk_ut`j z6tPV$?^}^;WUEP8Sf8RY04#L9z#yxl)6EAncU~!qA@hs?-{#)C60|2|Awe_kJ`imB zQ%mv+E=XnS{0uFe)Hv;&@p`&utz#El(DMiNU zOPMN5V$O#EFLJjoqG7`{=g`P1Q zuFhfmw}aY02UO_us(Gmug6{%?beINSHma_{Ih~gvBI#zOLB}~aYzS5YcLls}Q|%8E z9%PHY9*O5TC~yx)wc#D<#xkMV3p5-UCgjqY1t)^hp}*WMl`=L6EzBp2SKl~je8_yMiPZ> z<1MbgP-o9t#RFecGZMm3^g_>kP%|lG_a9N<;tLU85&E>UB>gH8TG43)( zPS=ycO|2}JBs)r}h)C~oI%rYoMRunoAoNTx%vk`Q%YdC8w`3hCWZ7XVbcQ5M$WckWax_vY-q_=QvH2W2%Psy z4jO}vZCN#UV+)Qh`MehkdD{gq?Xd!^aIrk!x)wrZp6fgDvM=HmFEB@-@2^5;rWXUq zxdrM_q-gnwrv#HVc+Tf+y_KkN|%pEq~vDA32^44QLng~T~l5i#^EkoG%y!SXH zVbwNCE0Pmg>8UZ2)>dV`pObvSnb-Eauy1cM{hs+Z;Xk8t6(EA@pxpQ`F+w|*)M6jz zC5U%JNu?I&Bv8YBQ=IXEDn&&f{BXLW6=SRYb~h~*$Oc9|B^No^&q?7Q0**fovZ+!z z2`X&kHO)b6ckd9bI%XNSu%01)8DTCIYKBaz$r%4C2rI;LKuBPP{4c<)XLZO^ZOv00 zFLbhNnp9VmA-NMN+R2X26n1sa4;)0z@Yd4J5ktG+`>7we_2?$jfBwRI%0&!Y%*N?`CU3<=h<^osbg8MlS!VyVxZQ?EGcn7~ z6kT=rJMP1104X%+xeE6}Cls25XO^L}_re-?QOI|0UdC%3s~HttEBQo<6L}v4+PhJ5 z4W2g^=Z`vkq1cSw9f~N^?wfB^wfK*0ZL^>e^e5Sz!9K4~WLwHcsLVKRsj>{hT|7?A zo5Y-_7KR$?b}!f3LD z5MT+cPlcYS!(fF_?>s{QiFLtK$a0BQZiq0QDuE7@qZ|fvIG1@7KQdHkt@2drgbgho zJqF!rVe*)5Jrbd`IVTw@mUZ({sQdD<9y`)75azYu?KC4urnPnf;tCrUc1N)hqO|bo zYpA+uO_E82#gaOUebeM<>c#Ie_%*kLBbm_}>LJE4cDJX|_P=~!C6Unu<6cui7YESk znq$XLhZ*M(XV`Bbrg*Efl-_cp-Ff~(-~Q(P`vgUNyd1!D40W5Kk?R<%r2Ilbbo0v7BY7&*V)v zL^)5iQX5-Ig1`AfKP}az1?0Hmco=@k`XJSt2}y;lmRTE(-vLR`&>QO7yh*wT=wQcC zGv11gHTRS%L6q57-im!kIIYwyeQT*>JC;(j`ep-=n=Bfwp*&P%i?xtNv{T3?*1x9W za^s+y+VjEC-xBz_TekX>nXr0+l{|Ms-?6U_2Aco5IwCdYFL!Pznr1scuwH@c)R2la z$Q6l@B`qZ$c&9==zkU)kN-#BI|EU|{y7qHf6FMtleOv3>ed@Qacm^htfF=8B0V_;NvuS`UfwC1I+k6yB*5 zkDQP&oM(<%OM)}kS_-zHw(|GmDC0T;)0SAbMX`Ql_;Swi7yRJk$EwcIj|?c80K~asRSaa%(Ca-;29~BA05N?Fb3aVt6-R!73SKq!*Sn6M@A;HCl>@;C z@BHOzw|bje2t+@;L*%qk|+A2yGJ%Je0!qIeP=Pkna{K9*BnvJ0Vm{ohN3oi?`x8utsv0?Q_SvU1GDb1kMgG;wYZ!$N zF1RBTd{^Fu zCvokaNP7Ic>hJKBSwcPU9aaF1RwheCx~>UhSqQaf5tu#6{DM)@sE zPM?{XSe5=jrb#`UR%(g$hT02D-vCX2i{S%?=Je}yP2TI-tzl<`7a*3ac-;3t?O4 zOLWpJ=ofLb9M&Zg-ZiZdq2-Y!hx%qIuWbnZ{Nh%`~_&YyElN6))MF&v48 zGcot@He?&EX#_=prF!O&3YIJ2G;(kd*Q`KS5&ZMZJJk zGmdR%FI~lkfh%9|Kve@9ZTNH&CtY9&Uz~@Fp#eU1tB1@fz*ByB6Qh~$9z3*sA_(t_ zT8fu2ENpJ^=2hZ8TK!jvPH*C`1^Y#P`Tg@Z^gB2mNW(`W%)DzNNZYX29({0^?e;gV z=knQ1cHnx~oy&w-o*7^7a?eNovPsI5z`@>hv%^~0 zq@?~xPjn;h-6xKrq67Q8OEKEZ_H=UX3o_KTt@Kik_8hXGj`b+STi4ecFPgQKSH{^2 zYi#Vm?^DrvFjIF?1H$%q&hW}bHB8DM ziff+SrQ_JlY|txQ>vB~(%X!bGM~lhP$V!9p{2GUG9$M#duBc_{pdqTPFe!|u?XY+@ zNp{N*%!q4>Hy#JiZMp|;nj3Kni(y}>`HP6SV&5qZ*G%_{;RS9JtR$z>u+S&zGYado zd_yCIAGv!(6k08@pT>0sn_M*_rY$JA@P<{g`C4fk5#G}Xs0!?RAH#;8ZdtBWsU_=% zN-?k0>?KcX>e|CCb5zeu6T<8eO@AyOo=_YkX5!hFtk*Bg433IMDUXQ!iJi`zmn&5h z@0PvcI4W0<#(w*JHdgaXS=P>jxrybeO5(w_;6nK^@hG*{ZJM%VNu2$_^5ysxHL5MO zIJEFsyVwF|8u7DSRb!_joJ&_MpIS-pHI;@IlVptk27R58LqF7CKndic*}1_TxN(6p z(it^~-cBBqgU#Y(17ygocjY!!a&6+cn#z=5$0n?uzl{ix+gvsD^}i%%#UNz{B@{4< zdoTMDnpjK4yL$_|cb&&**+N=X?zAh!mu!_QQ>3`jit6vf1mw-UPY=5M=+BeGqcHoE zsWp|kB)N!QnVXc8&s@LzTOa32&UTz)+GKxj z+WK1SY&a`$oWleT>50i~5Evz2={!~CZ+0D&7ou8GCO{bgd`R*qX`wak4aMc&ryM#L z&O;s5swf<*>De>H`QIc?lAMR;u9RgrUR-jWSl}YX1ntgV`@wf9p1ic;qwMLm;$Q7sC9Pyz`h9_x5A0C1@p^Eh=Z&?J)5xWvC#_=SSSCl72Ii?h18<+x;2y#YSrQ_7JGurM)((pLZE07WHFI$+dw0T1ihj z2ygqBYvW^?DU(pwQqN$(Ik^MAr_)nVg=Xak202Dm_lor!;nL^IfEsHv_=z>BVBxf> zFCtb7E6HdIsIKihz8SQt#RB5o>10h+p3s~)|MTIjLTwnveroHf)`USN-<(j^Rw-Ev zx>1||h|55xTTVc*26twb+_uYOEaNHy2ga_sD}xr{%OAkKmUqOmKf_wjeu!K84L;Hm zq=QYGvVCwi7o)cSYH`9v(^Sckz>va+LG&e3oWmCX;G?dqH{j2iAZxwahv?x|{`SRm zPGp*Dww}{@RS^%&%gk!erR_$_zQH$o*-fXpMzY|ks`x18TXC%xFuVaX^o8)GmUea2JN4Da4M1l?^XIOEzGSd#*fG=vPZZ@~sg*BbykIxYwo^TZ z@5zDt(w6r9=6=H6e)Uf%lt75g!OKbT3%JIEI%caLzQAV(!_U@7#~!D@Z~W|{wkav9 zeWiziYTDD`6}BxpwZi8|+QzOtgmr~P9he^W#p?bhmO+ctYD>vsYNt3B=eJ~?X5PRo zvRjw5Q-6NhqVKerI8&>zo`%z<+M~-jWuqNjtIlS%Cgt#p;y@vZjIy6fD>%PVyL6mI zb4^U&2{R>j7?lD91?uK7P>s3hkk9%+Vay_F=Umm3*FX^#O6uO@SP|Y4@)w1Hu&eQr zR$TYr^SmoqZ$;Yh{9J)*4#@sW-DgYq*vcc+HuSR*FpC71&$K@BQzO53^3}XuV}0(rR&66?F--^S;8WCEP^{msAPX zaDjQ9JZvp4MNsbUy@s3=#HjCnn}Y`H#xwKR)HRgWua+pgwb9V3fqm>!ajdZU8s@FZ z<+owy-oyKwV{vP+zyLwiLVtY!nV3w0(i`^fVXdv!t}I>=Z!Aba1bVDXFqc~cUVHE{ zn}>m;EkIb=tqO@WO6IO4f>}D+z*uoJRz+U8{CQLMW+_9wdWy;;u&wD#Q(*dz0km-U z&c`ksU6^M2=|pz-Nm?>JI$!pV_eH{Ni7rpYIPs=$iT7h1QvJcGld#Lf7_*jZn$#k0 z;LzIarlhKSWh0KB_u@IVz={yOQr@A3A*(6+WKKk-jE#u{`cf}D%*k6L7(*7z5E@-F zSs#B-4i+WQw~Q==Rfy0k@IAK^xNr@bs)YVFN+A`>#0s6*Zjate* zn|*$GN{&v!-5jf;yalABRW7`)>^cc>j#)-oxfImv%Iy3`(uYVlw^=y_KzfB*f*%nQ z7G*&>H0e<~*ReH$1)>yM@mxuo2hP!j2&Ap5<=$Wc#7lYn~XgB@US98V$l` zX!Vj@cNZZeoe8wf%t|pm35jzgLqMuQ7xAGX_%)aA;MkjN3|l(SQ;2I`N4Z;a*MOvz zC!gQ!EkPR#YDD0w8byCTtY1l5(D~9Q+dJ-R{zx{^D$WRa9I+Vo$g6qs#Y(N()*esq zoyi%Wrd2UPBj6}T`RZT_9#UnG6JCxI6njl<@_09(!=U~Yg-cE)!K%tZ-8MkIJ2(8q z0qf1*vc?Onb}1wB{i-6+33y!7PnO!aa2FEqxR-DU{(?ne!My(;WXk_8 zjr(WlW@i5ni^7P{%FgtkZjk>rbhEKD{eM(3M!SM2Ct0u3ij(62EpR)zN`yJN5^>No z(GNjSLF*_6EYQ%9M2kltN{UN>gNuuo5)vRveiioYe*WCL&S;)iX;_}Duf}-ly?S4J z@(hk!{3R>{7G)P)La=MfD~k)G&MPo8bijf~KtMx8Kp;9gV*bZ{5cu{7Wu3qiLx%WG z{?r1X5CaE|=vUw~lgh#Yf;Dw-11i|_D@&>>YpMzZ!y_TYzsunRQ9zXicJ7CLaVn2>KgfTt&9X1{^%4m{0ruOL1AK3^IWFpPLaSlAXS z0tmh?z|ict$eWtGlCRvD?WryLAU^vX(mn(~12KL-Jvs2B5PBHUJ3j#P0v00nTQAOU z3%G~~ApH~s4Ew)49=ieG3L|~+r`Tu?FXAb(%S&+nh(s6=%1KsPtPa&Nbf zD5{DJi!3b1zk_eKxebg1aNx*E3BbU^#3BF+2owzbnv%%iKdu-;@K?30!aw870YV6V zQ@J(_n@_nh-am){vwDO5P+y%Xyg-GT;C+v=W7fjNB3bl$1;4wBzu6DIN^iB4zjX1x zu0j_(H#fc8+`Zqwq5smt9bbQ`0EjegnN0zn;L{lVeqGBzj%MdB1p4T0V}D;P@&(ca zQj zh~GCs`lV-jQN<&l{p^}}{;5&jGa+5uxG0_mTD z{E-JvxLXZ3ObGgwAL#RiXW&PVxD)+>5g!8>Nc+6Z1(4uJ2f6RphVmssLMRM}6L2~B7CDInmssUtL_Dv(C#1s%ys@he>U9Zk=?`jjv44esF;Fg;*``s?tkZwIJe5xOFHG^x}@Pb>i zL&~esrSPcWPqVw+3DX`RHxH@Vv_AhR)v>Er531$xM12%)Qv-oqp0Fn03+pl?R(BgJ zloY#yO=Kb0qh7P@7^b%R*{t_wCWViB(c-ss#Yw*mX9a-u%2$38FXzRP_BARpYHGt# zy`+eu^zr4n`qqVcQT06!FO$Av&*093qWzVwd6BCpJgTD^aHib4=tkjtT-K@w7^Hu| z#gX@>nZUt+8CxrsWYp?Fcgo%X?W>hN%9@VlWZOLH8 z979bkA=~}Zq*CIQLRz_YaD1#-oKu5KP}^}{1)}HJq7UvPuP52w*sjZXCg{nXsPxq{ z*EN}P>xi3Iz*A#a+-;HJjTNcEGwhMbJ+Rbm&yUNk;X$arM-nDvhh4aX^pqs~)-}VH zoa5jRzqy8KAN!`QLf&*Cxt3?4&_ErzBH1*ERSu(XeM~AsQ`+~0OP8?+goT5=a5lrx zw|&7=)@A2%Iw7kT;BqV@MX}1pV6vGWI*W|PBMF?6>vQ5^ytwDcqE0Bna5V}P(LX47 zT0m8WGD@qtwHdRACV&m0`xH3ApGOkpXIux>jqoP_y}c7LsqJZ5F|Gt46!DgA8C1dA zkQs*(!N~%r0NYl z2?}J{;&aDjeM*XRQU;Seq$Ws=Fm0|pO_T6`IcCz@T&$W_9n1%>nQvgMM^vu3LI_GT z{BU}kO4l&11#fxBf=}2r6Z&B8Zq%~vG|@afBBvXN50(?Po-hjwfl|&RekNPt?P)!u zYeH4&+OouS%tPv{q^W7^2>Xj5x!Io+uI?9-#YVVTDxFyP3|<9{U_fU)D}$M5v38o{ z8V}M;qI2$2v#Ouuu4~rHHa2c7$5>rctwG214x2{8DQ0aO2UCr(dETSU2FKUMn#osE zFS;QLm^*OdC`g#}jMnOL?8W!9FYq)Ow(1g&y=C(}3?(J%PSFlEdtD(uA9y5*v{vvP59cPsOOhWWYLBWdTjqrFC_|lb>#^MQ# zBu2xro&pWvhdc-Wk{pGqKm7ztyqb-w<6jU`YfqQ@U>Cz(`k}oksXq9lEYI1P8*4cn zqJ^)yAQ2&&8TVS`F}i8Zl*I@881{iSoW-yvd}u;_I_AYxEXZtEJBdDo$T)|}x;-HGC{38=nUaR;%jP8oBuEM-^>4W3pq!AI2hK_?b*p?!NY}Rqgd|fhE+c zxymtZ!()9Y-joe#B5=843iWXz8jlmC&AViZ$nGvOR5*BC_E^a2%<3XDcL&zy16!5+4sJuH`~BBRZq~VKo1o<^H^&MZ=f&RDc(-*4mUd=7@=eZS+Xo++#QVO7 zysuUMxzgJK)mY$&aH$!$WG1>0?WVAdw-MgLBQkm4POnBZ3(W8)4=0e?;+8H{Tk98* z@#{3g>`GilD9bJe5?3rD+KSE*XH&|!+UDqGm*~4~yG3%9sOXr_ucT2|6UI~nB!>JA zIScIth`E7-7bTXPo^o6}YEgKgi<_nz`=@a5J9f=zT4cwn;$~prCYdsi1CA{!g@}L@ zyP&E=2U&FViT{yd(?a1FnczZ($=&eKcAXRtmXFEZ?*}OQmHlua7;^Ni_a2AoCY+&L z%hoC1bn%@FX^!hjG3vk)9_`9wLvuiXC>l#M4&0?~TvgfKa#GmZN~=A+#rwsbXf`j@ zcaFM{y(mzA^U*08&f{Sr=4`-58BGB^qWb!a!j-aLRx8## z(E3r_A!=ct;6kRCESdrL607TnGd^9C6s4hAG2f9#y(^Osooe{NX76zipas_}S{psY zT{>BZr7stHcYO>+cwl}gw|N6qUh%qXIj4D= zJ7DmGU(3Ah?L5y%*K_GS_N*2a_t=K?m`DFITNYcc;5@oUPn|IxbBSV3v$4liU39A4 zI4!4#xSt4-`0nTEe&i?F7Ugj{9)#{OUNQjOYNa5c1*%UTD*prX-TS!{sel}+{44b7MD%%W*6o>pS$UH;UlU8M9eNsP8#=0IrXkbBDX@-W-a;k;k@@Kh{ z(Vy&7YNWll2O!0l#=6XH1=}mh#zK~7k)EkUgr1+lUIPA2 zEpPHumIm;5W;@w#6j*O(ly9ycH>3&*9I`r_+G1me>boJMJZ7Syt&j?R_cHESP$exOxXXS<%3W(^Xx5!x$GO!GmF4 zN@Ytcl0lrnW#CGxI)l2JsI0_8_0`lSGy@*gSLS9$WmkrE=woUy6}}UV{|#3W zi>j9Uog)zBPmEVxx6o%c#&t|#VGupWB-tg{&#I81k)jj-QR*GRpwEvL^o;$tQ^7E{ zsU4@3X`;T-s{YHCipSJrw?w*=42Q2@45EKB#BD2Td@*wBSH;mTy`<79$A8WE1-s?8 zG;q|XMS!NLLw;wvf!|wO`0Z}^1O8}iWovXNI(B46Xe&@dE3OutZE4>n$uKmIj z{q3t(ChES%E}3X#z}nsHLc@1&jf&~x2>ZH}e#A_$0Jyc@s^2fSP*j5{7cgVdY`8!2Sgw@_Lk(YzS126JLqihW6+() zEcxF^*Wiih`OgF!@Q2<>XawZWEs%~R11}$^+RgVo*K~k(06Ev2b<|SdcU8!6#*EK-;UgjXU9;B ze<-T|qokS+-?G+7Z-0y8o%YmRm$nEDpeu#ux6@Y|EH)Gl-LANEqq5od zwCey*uEN%D`jX&`Mn^V27V7H2Lrd3yL)t-MDmS=XH&u$i?m`%dioN=`j}`H+b< z59NhSb@;PBWLW~OIP1ca*5O@tlq{UgN}vK|vHOId7=NDg9{aXXk2^)mP(xz@GksN1 z^>5Hmx|hO-v#-X7=~g9Fw!EUbTUOC|&n%A|rrT&8^pB-aHr*PmTirKC7xdzOE}SOp zZYulLTr9(TK3v^yyPcu2_GJA;HJr!un(chb9~b?Rt@>jzY0tQGpc_ve!Mo{wl=DiR zvO_Dn&MzZ_KNLh$Zt1*=z)|%uHa5G&$}AbW@L1pCLdxgFR%fCLT&w6?2{M{BzL>Z& zj{o-O&7EY@CvjqaLnO7){K6s0k?mz7;PV%{3mM;hkgn&vO;<7bU(dAkY_f$SJLo|1 z+vZRL;F2?Oh+3kT(9(znU*JTlu-Q?Lh?l_9xzMx2N{5DflXtPm>?hx>@NkCcEsjTt za!vm+WrYcO$in;4K22H2xdP8%bf-vzon~A+MfQjhTb6v~FhyX55eN$SlNET%8m8?U083l0QTb0S{I zYkRfH$51p4?aXwzOy{RL^!DaubcEj5a-3oH&$%3SmvM#mWRxZe*3Wp;AQDa1BJy$~ zlGbhQdHFW_5RHjF`RF%I6rXJhxc213A|7Q&e!tSr+2l?OeGwdJ4|KTiLo&kX#?zGZ z;fP49@dUQB-JN@mPM}D+YT!RcZ&XKLByM*MP4q1`u|N-owe+*6ohiIyjh@QC_r4Tx zQsNCrVt4W#q#FM;_-DSo2u=Z_`-;&P;Hc?dDw=M|<3arJRiS!C%A4B#cuRS@wJ`;X z>~)~}=DRSXy-EhTw@+-~hs`+oT1SGnvG8|q=j&goAllwYMb$lT$FME6s+5Kadiur1 zTXK|q-Kd5j2|>krN`*QmFE1pIp z449EsZ4#dgmAO*t91Ch|lr~AP`IwIMhHtF*2LN>{b~Y=CNRY$P&FPF20x4Z}l)4E; zFz;kl5yCB|)l;9^a1LG?42|th98#@z35;acHX%`(@g_9G4=(p&;>?k+xL~%}ibpG0L@lX$K7YV@qhEM^ zw(+(uROnTMtg7Gzezuzyq=D9ZC@57aXIdS8Mkp%RBPViJwjo|!_t>@BTbZ4$ z^sxvVq7{y3g?-oi?Y6>ZEPm5~E&7CE8B*7X+FQ~Int29xPXSf@U_UgE045YJqxwXvzt_|&|(}MZ)TZ5iCTrsLybt0si8?lG=FGqf*!QGioh-*+pM@=QO(C# zzwRTi4O&AYUWCmIiGGF*0TB;QB9m;qA?(2ILlJuD%TFb+C2^?k2exaj{nCkxadU{$( zJl&6HI2^JEv(gvPgl`N<62UoV%nUdx9Bm=if6nw)0nxHrxOF(8ziF8X@HHo526DI- ziKia`?w*4?J2|IQslhjF1~dsgHe$BgS;spc)R+0+qK6g|k827 zCpvch6&J=X7ByCXp`V_bZZ`;+6;IR+HwlEuHti;sp$99E_6-2}UUHM)_P&77`z*v3 z5C_K2+-){^PN>yK1p>9mOE?F4`mf|jmteP858R~FB`(YUEIAaf-(nPK&N{xXITvN`9GxC@E7P>pY`RoEH+w#BSzuIa_)T@(27!p zNE#ri&!i>)WMI5iBa83C&no3j&5;@#MSUbeV%eYkD8a&$waG1LIO#gnlp>mU+?1Ze z=Wky2z(&4Q(Bfq6_P^6k9b+6(mJ{+0W>x&k9!Q3p^bfUOu*d%)NpVT(5&oeKk9*R>TXnf2lo{ z3e20r_H_HGa&aA=_@*pkl0T2K!;~Y)W-3r|*jH6BrfU2BBvjv$e-`AfWD&r$F!#XU z_OpqFFRONFlDVJeu{Q>&c8WJ;b6s9aZjY76Qi6^q59YgBXb#Hr)8F_r_#Gso*|?I4 z%ll!i*btU4un#_T7M$)mdPkr0n8bFdaeD?GI!Syn^$aCfRu01Uw3%fQ%5;`iz6^SI zc~DS1cWl|~@ZOX{@J{Y!ZHOnpu69MIpy9r&+qH`S?rR=@UM>AhAo~*a7A5`7x@7;9 z+VWwC?V=~fAZt-u?^P-CYS>wB;m03V2r5k>@)ZyLsf{J?CL47($_JpJbP`bZM*=A; zu(@X$LWF2xvdQaic{O%s7iNK)S0mwh83rqI(PTEisc!ls%3wIr%1kqxS55Z)htxUB zWaXQE*kX)5%P7P|w_&=WyKf%GtCuBhjg*-dl$fd3i0B<85KF(tnn_Au`xU=m_US+~ zbwe@l8s$-&zo$7uv@BifO{!b44?iTkM9=<=E^|n^EzV-HvSXo>X}*5$#T!uiTHEOlTnW^sYAGXfUG$nui4MZSBgSWbS zMcTrGU*+#W4SYo474*SFg1z#hC7zwZ*%H&!cyx26oOPiFJZI&a)ncXW^{Me{aEyl_ znzIF%sj>$sKJFV*8rd@*np^Ws9eIg5D0=p^=>xsJv~(0{-$Tg~)~)47edLnG+HJJ7axIi}YBS34+{tCsckeORg1 zgr`!vaN-9_p)8Q`S`U<%Sn=WI8ya+2F}iDVZ|Jj>^pYFA)UVEBYoQv-v9V|HA=);e zz=wxuWA9|`(k4!iORf1E{X6d-7d(d$cQL6tDGQT@5w+!mH~Dj{l+oH~!m1o{YWe}( zo_LQh2q`_IHr&JH*BS&oF2vCmcbj1f*Yu`8)z6nQzHq{m(Efa)88G@j2D8(~d0mvt z!_uyLmD=r!dbXqkzm#!Dca|$ z$UL~Z3+B!hBy<-~SNLfn1$GKKE|d?ywu506)38;~vQ|$WsU#9eoe!pmBzHA6Q)(Xf z*jKJ!AkwyfKGC9xV8AC|qioR&`%5?md@F4#y*~Vlltn?T|DXTCIoSYg0I0DQrhot@ ztCAzs)Y$~$2w<=`HB*CVF@sn**jT{ajF_xqju0a!=<~>P_rIH#P&;uWCkTK+oDa+f z=49h$;|6iDv$JzCvvJe2vC%)r$wN*5TZo#Yk-a^{6u>HJWCMddgHV&uVv%&Vu`xEX zv-<~&nuR3{@O=IY(?@^?#1Zy9HGrK3!~x>u1#@w-GlN;U{tfnjAp+zqT_AveP_arw zZ6U^v5LZTkwY`y(1q@;YaQQz^;9}um0Wo3<3H_6of0DrYeM#Nf%%)<-l|dCzLoC_`RYY5qCErI?4pAi#i`>hLBkLm<1W6myUI60w7rT0Tvh=$V-z zHJ6DHel^oDOpHO4$VKFlnxrn63ZYfz%ih(#?`%}uGrY~sRKH;~-g9=sk9#4)9<{UY>UHB@WqYl<-v@(rf<&WgCL14R|m+ ziC(&)p%jnZ>qvZP&O|;6P5tk}V*SHBYQLyQ?IZ%n)c&I+nCLa6kvDaS^MckHMxXknN?Pj);eQ@xE7BpAhd~8SIs? zE_*Be=}=i5)5FvnODiTi+|(6-Zp((vCDFxv!-+dxr>sGe+amTSCwc8_4@TGT-IlIQ z5~O=Fvst2^m6}LNHKO+uk>6RW0^X%9avh!rT@U~a06q7q zK%q_m4&LYcRsIP#g984S_5W#z19SyI5@0q-QE?t#Nih(ZM}h+^E+HYx$-~VqDbCBo z$;B%K`0pXlv;51#2LR#$|BDa*KM0XB%I;F zUwIMQNg!_c18p&24VRNJIrY7j5)V0!au>mm!G{f5geg5cN-0DxPzNstUwDusN%Vqu z)_mTrOCzcWV~Fr%Y2$?CTFAjrprND7T;Q(P$M1c#+AeVZMiVdoXJuec YMvhKyj?cB=;O1m!!=$B^P?W^{A5eOcdjJ3c literal 0 HcmV?d00001 From 663ce1d6c06aaf3ae2f91e763af1bdd39fc4609f Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 9 Jul 2025 12:30:41 -0700 Subject: [PATCH 241/277] Bump RTD Ubuntu version [skip ci] --- .readthedocs.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.readthedocs.yaml b/.readthedocs.yaml index b6421ddd6..6f2cf5dd9 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -6,7 +6,7 @@ version: 2 # Set the OS, Python version and other tools you might need build: - os: ubuntu-22.04 + os: ubuntu-24.04 tools: python: "3.12" # You can also specify other tool versions: From 4baa4c7eeae8e0b6a7ace4dde242ac165bcd59d9 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 9 Jul 2025 15:22:13 -0700 Subject: [PATCH 242/277] Add support for spell checking documentation [skip ci] --- CHANGELOG.md | 2 +- docs/Makefile | 12 ++- docs/source/arrays.rst | 2 +- docs/source/conf.py | 11 ++- docs/source/configuration.rst | 2 +- docs/source/faq.rst | 12 +-- docs/source/index.inc | 2 +- docs/source/issues.rst | 2 +- docs/source/spelling_wordlist.txt | 124 ++++++++++++++++++++++++++++++ docs/source/versions.rst | 4 +- docs/source/zfpcmd.rst | 16 ++-- 11 files changed, 162 insertions(+), 27 deletions(-) create mode 100644 docs/source/spelling_wordlist.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index c12f216b0..7903a4210 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -118,7 +118,7 @@ moved relative to prior versions. - #86: C++ compiler warns when `__STDC_VERSION__` is undefined. - #87: `CXXFLAGS` is misspelled in `cfp/src/Makefile`. - #98: `zfp_stream_maximum_size()` underestimates size in reversible mode. -- #99: Incorrect `private_view` reads due to missing writeback. +- #99: Incorrect `private_view` reads due to missing write-back. - #109: Unused CPython array is incompatible with PyPy. - #112: PGI compiler bug causes issues with memory alignment. - #119: All-subnormal blocks may cause floating-point overflow. diff --git a/docs/Makefile b/docs/Makefile index 22a0d97ec..987761f1f 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -3,18 +3,26 @@ # You can set these variables from the command line. SPHINXOPTS = -SPHINXBUILD = /usr/bin/python3 -msphinx +SPHINXBUILD = /usr/bin/python3 -m sphinx SPHINXPROJ = zfp SOURCEDIR = source BUILDDIR = build +# Needed for spell checking on macOS +PYENCHANT_LIBRARY_PATH = /opt/homebrew/lib/libenchant-2.dylib + +# Build HTML by default all: @$(MAKE) html -# Put it first so that "make" without argument is like "make help". +# List targets help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) +# Run spell checker +spell: + @PYENCHANT_LIBRARY_PATH=$(PYENCHANT_LIBRARY_PATH) $(SPHINXBUILD) -b spelling -d "$(BUILDDIR)/doctrees" "$(SOURCEDIR)" "$(BUILDDIR)/spelling" + .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new diff --git a/docs/source/arrays.rst b/docs/source/arrays.rst index 31962e822..2a0999085 100644 --- a/docs/source/arrays.rst +++ b/docs/source/arrays.rst @@ -502,7 +502,7 @@ with only a few differences: - All methods other than those that specify array-wide settings, such as compression mode and parameters, array dimensions, and array contents, are :code:`const` qualified. There are, thus, no methods for obtaining - a writeable reference, pointer, or iterator. Consequently, one may not + a writable reference, pointer, or iterator. Consequently, one may not initialize such arrays one element at a time. Rather, the user initializes the whole array by passing a pointer to uncompressed data. diff --git a/docs/source/conf.py b/docs/source/conf.py index 52764249a..21bccdc5d 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -27,6 +27,8 @@ # # needs_sphinx = '1.0' +import sys + # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. @@ -36,6 +38,10 @@ 'sphinxfortran.fortran_domain' ] +# Require sphinxcontrib.spelling only when running spell checker. +if 'spelling' in sys.argv: + extensions += ['sphinxcontrib.spelling'] + # Ensure rasterization of vector graphics uses sufficient DPI image_converter_args = ['-density', '300', '-geometry', '50%'] @@ -164,7 +170,7 @@ # author, documentclass [howto, manual, or own class]). latex_documents = [ (master_doc, 'zfp.tex', u'zfp Documentation', - u'\shortstack[l]{Peter Lindstrom\\\\Danielle Asher}', 'manual'), + u'\\shortstack[l]{Peter Lindstrom\\\\Danielle Asher}', 'manual'), ] @@ -188,6 +194,3 @@ author, 'zfp', 'One line description of project.', 'Miscellaneous'), ] - - - diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst index 3e7da15f3..5b00fb845 100644 --- a/docs/source/configuration.rst +++ b/docs/source/configuration.rst @@ -122,7 +122,7 @@ or written is one byte, and endianness does not matter. Such |zfp| streams are portable. .. warning:: - For compressed streams to be portabile across platforms with different byte + For compressed streams to be portable across platforms with different byte order, |zfp| must be built with a word size of 8 bits. When using the |zfp| :ref:`bitstream API `, it is possible to write diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 998a5bba9..358ffd096 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -490,7 +490,7 @@ Additional benefits of this framework include: - No need to perform surgery on |zfp|. - Support for arbitrarily small |zfp| error tolerances (see :ref:`Q17 `) and even lossless compression in the limit. -- Easy integratation with current file formats and I/O libraries. +- Easy integration with current file formats and I/O libraries. ------------------------------------------------------------------------------- @@ -733,8 +733,8 @@ the number of blocks *bx* |times| *by*:: bitsize = (4 * bx) * (4 * by) * rate -where *nx* |leq| 4 |times| bx < *nx* + 4 and -*ny* |leq| 4 |times| *by* < *ny* + 4. When amortizing bitsize over the +where *nx* |leq| 4 |times| *bx* < *nx* + 4 and +*ny* |leq| 4 |times| *by* < *ny* + 4. When amortizing *bitsize* over the *nx* |times| *ny* values, a slightly higher rate than requested may result. Third, to support updating compressed blocks, as is needed by |zfp|'s @@ -770,8 +770,8 @@ uncompressed array to avoid having to allocate separate storage for the compressed stream. |zfp| does allow for the possibility of such in-place compression, but with several caveats and restrictions: - 1. A bitstream must be created whose buffer points to the beginning of - uncompressed (and to be compressed) storage. + 1. A :c:type:`bitstream` must be created whose buffer points to the beginning + of uncompressed (and to be compressed) storage. 2. The array must be compressed using |zfp|'s low-level API. In particular, the data must already be partitioned and organized into contiguous blocks @@ -1308,7 +1308,7 @@ resulting from |zfp|, as detailed in the following publications: SIAM Journal on Scientific Computing, 2019. #. D. Hammerling, A. Baker, A. Pinard, P. Lindstrom, "`A Collaborative Effort to Improve Lossy Compression Methods for Climate Data `__," - 5th International Workshop on Data Analysis and Reduction for Big Scientific Data, 2019. + 5\ :sup:`th` International Workshop on Data Analysis and Reduction for Big Scientific Data, 2019. #. A. Fox, J. Diffenderfer, J. Hittinger, G. Sanders, P. Lindstrom. "`Stability Analysis of Inline ZFP Compression for Floating-Point Data in Iterative Methods `__," SIAM Journal on Scientific Computing, 2020. diff --git a/docs/source/index.inc b/docs/source/index.inc index e6e2232f5..ede2b27cf 100644 --- a/docs/source/index.inc +++ b/docs/source/index.inc @@ -44,7 +44,7 @@ and speed of access: The top 32 bits of a 44-bit base offset are stored, with the 12 least significant bits of this base set to zero. Four unsigned 16-bit deltas from the base offset complete the representation. The default for - variable-rate arrays, this index offers a good tradeoff between storage, + variable-rate arrays, this index offers a good trade-off between storage, offset range, and speed. * :cpp:class:`hybrid8`: Eight consecutive offsets are encoded together diff --git a/docs/source/issues.rst b/docs/source/issues.rst index 031be234b..fdef39d26 100644 --- a/docs/source/issues.rst +++ b/docs/source/issues.rst @@ -296,7 +296,7 @@ the exponent of the largest (in magnitude) value within a block, but produces unspecified behavior if that value is not finite. |zfp| currently has no independent mechanism for handling fill values. Ideally -such special values would be signalled separately, e.g., using a bit mask, +such special values would be signaled separately, e.g., using a bit mask, and then replaced with zeros to ensure that they both compress well and do not pollute actual data. diff --git a/docs/source/spelling_wordlist.txt b/docs/source/spelling_wordlist.txt new file mode 100644 index 000000000..942b01221 --- /dev/null +++ b/docs/source/spelling_wordlist.txt @@ -0,0 +1,124 @@ +ABI +accessor +accessors +AoS +asinh +atomics +bitwise +blockwise +cfp +chroma +cmake +CMake +cmocka +codec +codecs +codestream +compressibility +Conda +const +CPython +curvilinear +de +deallocate +deallocated +deallocates +deallocating +decompressor +decorrelate +decorrelated +decorrelating +decorrelation +denormals +dereference +dereferencing +deserialization +deserialize +deserialized +deserializing +destructor +destructors +dimensionalities +dimensionality +endian +endianness +enum +enums +equidimensional +executables +exponentiating +fortran +googletest +grayscale +Hadamard +headerless +libzfp +linearizing +lossy +losslessly +lvalue +macOS +mallocs +multicore +multithreaded +multithreading +mutator +mutators +namespace +namespaces +NaN +negabinary +optimality +ORed +partitioner +piecewise +pointwise +postfix +pre +precisions +prepended +prepending +preprocessor +priori +programmatically +quantized +radix +redistributions +representable +reStructuredText +roundoff +rvalue +scikit +significand +significands +SoA +strided +struct +structs +subarray +subarrays +subdirectories +suboptimal +subnormals +subsampling +subsetting +substream +templated +typedefs +uncategorized +unpromoted +zag +zFORp +zfp +zfPy +zig +Asher +Diffenderfer +Haiying +Hammerling +Hittinger +Kuffuor +Magri +Osei +Pinard +Xu diff --git a/docs/source/versions.rst b/docs/source/versions.rst index 0824e55cd..d82c3f229 100644 --- a/docs/source/versions.rst +++ b/docs/source/versions.rst @@ -109,7 +109,7 @@ moved relative to prior versions. - #86: C++ compiler warns when ``__STDC_VERSION__`` is undefined. - #87: ``CXXFLAGS`` is misspelled in ``cfp/src/Makefile``. - #98: ``zfp_stream_maximum_size()`` underestimates size in reversible mode. -- #99: Incorrect ``private_view`` reads due to missing writeback. +- #99: Incorrect ``private_view`` reads due to missing write-back. - #109: Unused CPython array is incompatible with PyPy. - #112: PGI compiler bug causes issues with memory alignment. - #119: All-subnormal blocks may cause floating-point overflow. @@ -328,7 +328,7 @@ not backward compatible with previous versions of |zfp|. - Support for 31-bit and 63-bit integer data, as well as shorter integer types. - New examples for evaluating the throughput of the (de)compressor and for - compressing grayscale images in the pgm format. + compressing grayscale images in the ``pgm`` format. - Frequently asked questions. **Changed** diff --git a/docs/source/zfpcmd.rst b/docs/source/zfpcmd.rst index 3f93a41e5..6b6b5582a 100644 --- a/docs/source/zfpcmd.rst +++ b/docs/source/zfpcmd.rst @@ -115,10 +115,10 @@ General options Evaluate and print the following error statistics: - * rmse: The root mean square error. - * nrmse: The root mean square error normalized to the range. - * maxe: The maximum absolute pointwise error. - * psnr: The peak signal to noise ratio in decibels. + * ``rmse``: The root mean square error. + * ``nrmse``: The root mean square error normalized to the range. + * ``maxe``: The maximum absolute pointwise error. + * ``psnr``: The peak signal to noise ratio in decibels. Input and output ^^^^^^^^^^^^^^^^ @@ -246,11 +246,11 @@ Examples * :code:`-i file` : read uncompressed file and compress to memory * :code:`-z file` : read compressed file and decompress to memory - * :code:`-i ifile -z zfile` : read uncompressed ifile, write compressed zfile - * :code:`-z zfile -o ofile` : read compressed zfile, write decompressed ofile - * :code:`-i ifile -o ofile` : read ifile, compress, decompress, write ofile + * :code:`-i ifile -z zfile` : read uncompressed ``ifile``, write compressed ``zfile`` + * :code:`-z zfile -o ofile` : read compressed ``zfile``, write decompressed ``ofile`` + * :code:`-i ifile -o ofile` : read ``ifile``, compress, decompress, write ``ofile`` * :code:`-i file -s` : read uncompressed file, compress to memory, print stats - * :code:`-i - -o - -s` : read stdin, compress, decompress, write stdout, print stats + * :code:`-i - -o - -s` : read ``stdin``, compress, decompress, write ``stdout``, print stats * :code:`-f -3 100 100 100 -r 16` : 2x fixed-rate compression of 100 |times| 100 |times| 100 floats * :code:`-d -1 1000000 -r 32` : 2x fixed-rate compression of 1,000,000 doubles * :code:`-d -2 1000 1000 -p 32` : 32-bit precision compression of 1000 |times| 1000 doubles From e1c8637c7527ef17b9cb3a5f39478bc4f9ebbf73 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 11 Aug 2025 18:40:20 +0000 Subject: [PATCH 243/277] Bump actions/checkout from 4 to 5 Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v4...v5) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '5' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 8fa09a3b5..9b0d05f81 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -11,7 +11,7 @@ jobs: if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Setup Python uses: actions/setup-python@v5 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index be336cb47..85fd5140b 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -7,7 +7,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout Zfp - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Setup Python uses: actions/setup-python@v5 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index f68f1353d..f41311b54 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -7,7 +7,7 @@ jobs: runs-on: macos-latest steps: - name: Checkout Zfp - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Setup Python uses: actions/setup-python@v5 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index dc328c152..381ed7fae 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -51,7 +51,7 @@ jobs: target: all steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - name: Setup Python uses: actions/setup-python@v5 From 94acfd207dc11745f1c8e59b98571ab2973f572d Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 11 Aug 2025 11:46:38 -0700 Subject: [PATCH 244/277] Fix documentation of relative error bound --- docs/source/faq.rst | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/source/faq.rst b/docs/source/faq.rst index 358ffd096..92cd8b270 100644 --- a/docs/source/faq.rst +++ b/docs/source/faq.rst @@ -828,8 +828,11 @@ floating-point values and then losslessly compressing the result. The *q* least significant bits of *n*-bit floating-point numbers (*n* = 32 for floats and *n* = 64 for doubles) are truncated by |zfp| by specifying a maximum precision of *p* = *n* |minus| *q*. The resulting point-wise relative -error is then at most 2\ :sup:`q - 23` (for floats) or 2\ :sup:`q - 52` -(for doubles). +error is then at most 2\ :sup:`3 d + q - 23` for floats and +2\ :sup:`3 d + q - 52` for doubles, where *d* is the dimensionality of +the data (1 |leq| *d* |leq| 4). Expressed in terms of *p*, the relative error +is at most 2\ :sup:`3 (d + 3) - p` for floats and 2\ :sup:`3 (d + 4) - p` +for doubles. .. note:: For large enough *q*, floating-point exponent bits will be discarded, @@ -838,15 +841,21 @@ error is then at most 2\ :sup:`q - 23` (for floats) or 2\ :sup:`q - 52` for subnormals; however, such values are likely too small for relative errors to be meaningful. +.. warning:: + For the bound to hold, |zfp| must be modified to avoid the non-reversible + code path when less than full precision is used. This issue will be + addressed in the next |zfp| release. + To bound the relative error, set the expert mode parameters to:: - minbits = 0 - maxbits = 0 + minbits = ZFP_MIN_BITS + maxbits = ZFP_MAX_BITS maxprec = p minexp = ZFP_MIN_EXP - 1 = -1075 For example, using the |zfpcmd| command-line tool, set the parameters using -:option:`-c` :code:`0 0 p -1075`. +:option:`-c` :code:`0 0 p -1075` (|zfpcmd| will replace the zeros with +defaults). Note that while the above approach respects the error bound when the above conditions are met, it uses |zfp| for a purpose it was not designed From c227e17acbf97db2abaaba839dc985a18c6acbfd Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 6 Sep 2025 18:31:07 -0700 Subject: [PATCH 245/277] Revert to macos-14 for gcc compatibility --- .github/workflows/tests.yml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 381ed7fae..9f8278b96 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -38,7 +38,12 @@ jobs: target: all architecture: arm64 - - os: macos-latest +# macos-latest = macos-15 is currently incompatible with Homebrew gcc; +# see https://github.com/actions/runner-images/issues/12745. +# For now, use macos-14. + +# - os: macos-latest + - os: macos-14 cxx_compiler: g++-12 c_compiler: gcc-12 omp: ON @@ -70,7 +75,7 @@ jobs: run: sudo apt-get update; sudo apt-get install -y libomp5 libomp-dev - name: Setup OpenMP (MacOS) - if: ${{matrix.os == 'macos-latest'}} + if: ${{(matrix.os == 'macos-latest' || matrix.os == 'macos-14')}} run: | brew install libomp echo "CC=$(brew --prefix llvm)/bin/clang" >> $GITHUB_ENV From c2dd2966f6bab18e352b0fa29c3c5f27836580f8 Mon Sep 17 00:00:00 2001 From: John E Date: Wed, 3 Sep 2025 16:06:30 -0400 Subject: [PATCH 246/277] fix typo ? --- setup.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/setup.py b/setup.py index 90b37865c..b4bc530ec 100644 --- a/setup.py +++ b/setup.py @@ -20,13 +20,13 @@ def __repr__(self): long_description="zfp is a compressed format for representing multidimensional floating-point and integer arrays. zfp provides compressed-array classes that support high throughput read and write random access to individual array elements. zfp also supports serial and parallel compression of whole arrays using both lossless and lossy compression with error tolerances. zfp is primarily written in C and C++ but also includes Python and Fortran bindings.", ext_modules=[ Extension( - "zfpy", + "zfpy", sources=["python/zfpy.pyx"], include_dirs=["include", str(NumpyImport())], - libraries=["zfp"], + libraries=["zfp"], library_dirs=["build/lib64", "build/lib/Release"], language_level=3, - lanugage="c", + language="c", ), ], classifiers=[ From 4470ca6bceeb85f4fed503b26511f8c05d85a0c4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 8 Sep 2025 13:14:40 +0000 Subject: [PATCH 247/277] Bump actions/setup-python from 5 to 6 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 9b0d05f81..92a00d04a 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -14,7 +14,7 @@ jobs: uses: actions/checkout@v5 - name: Setup Python - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 85fd5140b..093d9b870 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -10,7 +10,7 @@ jobs: uses: actions/checkout@v5 - name: Setup Python - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index f41311b54..7b955ebc1 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -10,7 +10,7 @@ jobs: uses: actions/checkout@v5 - name: Setup Python - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 9f8278b96..e3036acd6 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -59,7 +59,7 @@ jobs: - uses: actions/checkout@v5 - name: Setup Python - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: '3.x' architecture: ${{ matrix.architecture || 'x64' }} From 0596d4964ac1692f88ac43b1b3d1e66c66696b6b Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 3 Oct 2025 08:50:14 -0700 Subject: [PATCH 248/277] Update GoogleTest version to latest --- tests/CMakeLists.txt.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/CMakeLists.txt.in b/tests/CMakeLists.txt.in index 46e7b0880..d476982a6 100644 --- a/tests/CMakeLists.txt.in +++ b/tests/CMakeLists.txt.in @@ -1,4 +1,4 @@ -cmake_minimum_required(VERSION 2.8.7) +cmake_minimum_required(VERSION 3.9) project(googletest-download NONE) @@ -6,7 +6,7 @@ include(ExternalProject) ExternalProject_Add( googletest GIT_REPOSITORY https://github.com/google/googletest.git - GIT_TAG e2239ee6043f73722e7aa812a459f54a28552929 #703bd9caab50b139428cea1aaff9974ebee5742e + GIT_TAG main SOURCE_DIR "${ZFP_BINARY_DIR}/tests/googletest-src" BINARY_DIR "${ZFP_BINARY_DIR}/tests/googletest-build" CONFIGURE_COMMAND "" From 40a0c664a286f8de06e556ddec8c04c36e059e08 Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 3 Oct 2025 10:41:08 -0700 Subject: [PATCH 249/277] Update cmake version on LC; remove Lassen --- tests/gitlab/dane-jobs.yml | 8 ++++---- tests/gitlab/gitlab-ci.yml | 6 ++++-- tests/gitlab/matrix-jobs.yml | 17 +++++++++++++++++ tests/gitlab/matrix-templates.yml | 12 ++++++++++++ 4 files changed, 37 insertions(+), 6 deletions(-) create mode 100644 tests/gitlab/matrix-jobs.yml create mode 100644 tests/gitlab/matrix-templates.yml diff --git a/tests/gitlab/dane-jobs.yml b/tests/gitlab/dane-jobs.yml index b4405c2a3..77b05bac8 100644 --- a/tests/gitlab/dane-jobs.yml +++ b/tests/gitlab/dane-jobs.yml @@ -4,7 +4,7 @@ cpp_gnu-10.3.1_build: variables: - ci_cmake: "cmake/3.14.5" + ci_cmake: "cmake/3.23.1" ci_cxx_cmp: "g++" ci_c_cmp: "gcc" ci_cmp_mod: "gcc/10.3.1" @@ -18,7 +18,7 @@ cpp_gnu-10.3.1_test: cpp_clang-14.0.6_build: variables: - ci_cmake: "cmake/3.14.5" + ci_cmake: "cmake/3.23.1" ci_cxx_cmp: "clang++" ci_c_cmp: "clang" ci_cmp_mod: "clang/14.0.6" @@ -32,7 +32,7 @@ cpp_clang-14.0.6_test: cpp_intel-2022.1.0_build: variables: - ci_cmake: "cmake/3.14.5" + ci_cmake: "cmake/3.23.1" ci_cxx_cmp: "icpc" ci_c_cmp: "icc" ci_cmp_mod: "intel/2022.1.0" @@ -51,7 +51,7 @@ cpp_intel-2022.1.0_test: c_gnu-10.3.1_build: variables: - ci_cmake: "cmake/3.14.5" + ci_cmake: "cmake/3.23.1" ci_c_cmp: "gcc" ci_cmp_mod: "gcc/10.3.1" extends: [.c, .dane_build_cpu] diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 8fabca5fd..1912f5ce5 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -139,8 +139,10 @@ include: file: 'id_tokens.yml' # - local: tests/gitlab/pascal-templates.yml # - local: tests/gitlab/pascal-jobs.yml - - local: tests/gitlab/lassen-templates.yml - - local: tests/gitlab/lassen-jobs.yml +# - local: tests/gitlab/lassen-templates.yml +# - local: tests/gitlab/lassen-jobs.yml +# - local: tests/gitlab/matrix-templates.yml +# - local: tests/gitlab/matrix-jobs.yml - local: tests/gitlab/dane-templates.yml - local: tests/gitlab/dane-jobs.yml # - local: tests/gitlab/quartz-templates.yml diff --git a/tests/gitlab/matrix-jobs.yml b/tests/gitlab/matrix-jobs.yml new file mode 100644 index 000000000..535c9197d --- /dev/null +++ b/tests/gitlab/matrix-jobs.yml @@ -0,0 +1,17 @@ +############ +# CUDA GPU # +############ + +cuda-11.8.0_build: + variables: + ci_cmake: "cmake/3.23.1" + ci_cmp_mod: "cuda/11.8.0" + ci_gcc_mod: "gcc/8.3.1" + extends: [.cuda, .matrix_build_gpu] + needs: [] + +cuda-11.8.0_test: + variables: + ci_test_regex: "Cuda" + extends: [.matrix_test_gpu] + needs: [cuda-11.8.0_build] diff --git a/tests/gitlab/matrix-templates.yml b/tests/gitlab/matrix-templates.yml new file mode 100644 index 000000000..95180b879 --- /dev/null +++ b/tests/gitlab/matrix-templates.yml @@ -0,0 +1,12 @@ +.matrix_job: + tags: + - batch + - matrix + +.matrix_build_gpu: + extends: [.build_gpu, .matrix_job] + +.matrix_test_gpu: + variables: + ci_test_regex: "." + extends: [.test_gpu, .matrix_job] From c03a7e14e1dbed90897e2d7d2cfa25f09c0e0d7c Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 4 Oct 2025 10:59:11 -0700 Subject: [PATCH 250/277] Require C++17 for GoogleTest; bump cmocka to 1.1.8 --- tests/CMakeLists.txt | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 4134a2734..ca8e24972 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -17,7 +17,8 @@ if(BUILD_TESTING OR BUILD_TESTING_FULL) endif() if(BUILD_TESTING_FULL) - set(CMAKE_CXX_STANDARD 11) + # C++17 needed as of GoogleTest 1.17.0 + set(CMAKE_CXX_STANDARD 17) # CMAKE_SH-NOTFOUND needed for mingw builds if(MINGW) @@ -25,20 +26,20 @@ if(BUILD_TESTING_FULL) list(APPEND GTEST_ARGS "-DCMAKE_SH=CMAKE_SH-NOTFOUND") endif() - # clone cmocka 1.1.0 into /build - list(APPEND CMOCKA_ARGS "-DWITH_STATIC_LIB=ON;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") + # clone cmocka into /build (WITH_STATIC_LIB replaced by BUILD_SHARED_LIBS in cmocka 1.1.6) + list(APPEND CMOCKA_ARGS "-DBUILD_SHARED_LIBS=OFF;-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER};-DUNIT_TESTING=OFF") include(ExternalProject) ExternalProject_Add( cmocka_cloned GIT_REPOSITORY https://gitlab.com/cmocka/cmocka.git - GIT_TAG cmocka-1.1.5 + GIT_TAG cmocka-1.1.8 SOURCE_DIR "${CMAKE_BINARY_DIR}/cmocka-src" BINARY_DIR "${CMAKE_BINARY_DIR}/cmocka-build" CMAKE_ARGS "${CMOCKA_ARGS}" INSTALL_COMMAND "" - STEP_TARGETS build - EXCLUDE_FROM_ALL TRUE + STEP_TARGETS build + EXCLUDE_FROM_ALL TRUE ) ExternalProject_Get_Property(cmocka_cloned source_dir binary_dir) @@ -49,7 +50,7 @@ if(BUILD_TESTING_FULL) if(MSVC) set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka-static.lib") else() - set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka-static.a") + set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka.a") endif() set_property(TARGET cmocka PROPERTY @@ -76,8 +77,7 @@ if(BUILD_TESTING_FULL) if(MSVC) add_compile_options(/wd4244) endif() - - + add_subdirectory(utils) add_subdirectory(src) From 8b53d593609e24ae63214785af0b90aabbac6a0a Mon Sep 17 00:00:00 2001 From: lindstro Date: Sat, 4 Oct 2025 13:25:34 -0700 Subject: [PATCH 251/277] Fix cmocka.lib path --- tests/CMakeLists.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index ca8e24972..7fb874701 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -48,7 +48,7 @@ if(BUILD_TESTING_FULL) # choose proper library path & extension if(MSVC) - set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka-static.lib") + set(IMPORTED_LOCATION_PATH "${binary_dir}/src/${CMAKE_BUILD_TYPE}/cmocka.lib") else() set(IMPORTED_LOCATION_PATH "${binary_dir}/src/libcmocka.a") endif() From cbf14c51bced44baf7d65891b0e61cecf3c04255 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 6 Oct 2025 10:01:06 -0700 Subject: [PATCH 252/277] Include stdint.h for cmocka --- tests/src/decode/zfpDecodeBlockBase.c | 1 + tests/src/encode/zfpEncodeBlockBase.c | 1 + tests/src/endtoend/zfpEndtoendBase.c | 1 + tests/src/execPolicy/testCuda.c | 1 + tests/src/execPolicy/testOmp.c | 1 + tests/src/execPolicy/testOmpInternal.c | 1 + tests/src/inline/testBitstream.c | 1 + tests/src/inline/testBitstreamSmallWsize.c | 1 + tests/src/inline/testBitstreamStrided.c | 1 + tests/src/misc/testZfpField1d.c | 1 + tests/src/misc/testZfpField1f.c | 1 + tests/src/misc/testZfpField2d.c | 1 + tests/src/misc/testZfpField2f.c | 1 + tests/src/misc/testZfpField3d.c | 1 + tests/src/misc/testZfpField3f.c | 1 + tests/src/misc/testZfpField4d.c | 1 + tests/src/misc/testZfpField4f.c | 1 + tests/src/misc/testZfpHeader.c | 1 + tests/src/misc/testZfpPromote.c | 1 + tests/src/misc/testZfpStream.c | 1 + 20 files changed, 20 insertions(+) diff --git a/tests/src/decode/zfpDecodeBlockBase.c b/tests/src/decode/zfpDecodeBlockBase.c index caeb90a36..7691d7aaf 100644 --- a/tests/src/decode/zfpDecodeBlockBase.c +++ b/tests/src/decode/zfpDecodeBlockBase.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/encode/zfpEncodeBlockBase.c b/tests/src/encode/zfpEncodeBlockBase.c index 23917c622..5d72a6fc3 100644 --- a/tests/src/encode/zfpEncodeBlockBase.c +++ b/tests/src/encode/zfpEncodeBlockBase.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/endtoend/zfpEndtoendBase.c b/tests/src/endtoend/zfpEndtoendBase.c index 5069824f9..76ddf3c0c 100644 --- a/tests/src/endtoend/zfpEndtoendBase.c +++ b/tests/src/endtoend/zfpEndtoendBase.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/execPolicy/testCuda.c b/tests/src/execPolicy/testCuda.c index 83d2d1fc4..99640109c 100644 --- a/tests/src/execPolicy/testCuda.c +++ b/tests/src/execPolicy/testCuda.c @@ -2,6 +2,7 @@ #include #include +#include #include #include diff --git a/tests/src/execPolicy/testOmp.c b/tests/src/execPolicy/testOmp.c index 29ab5d57b..4fffffbd1 100644 --- a/tests/src/execPolicy/testOmp.c +++ b/tests/src/execPolicy/testOmp.c @@ -2,6 +2,7 @@ #include #include +#include #include #include diff --git a/tests/src/execPolicy/testOmpInternal.c b/tests/src/execPolicy/testOmpInternal.c index dfa89eb57..1436a6eca 100644 --- a/tests/src/execPolicy/testOmpInternal.c +++ b/tests/src/execPolicy/testOmpInternal.c @@ -4,6 +4,7 @@ #include #include +#include #include #include diff --git a/tests/src/inline/testBitstream.c b/tests/src/inline/testBitstream.c index 29fe8151b..d6e559159 100644 --- a/tests/src/inline/testBitstream.c +++ b/tests/src/inline/testBitstream.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/inline/testBitstreamSmallWsize.c b/tests/src/inline/testBitstreamSmallWsize.c index 290dd2924..e84c72af1 100644 --- a/tests/src/inline/testBitstreamSmallWsize.c +++ b/tests/src/inline/testBitstreamSmallWsize.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/inline/testBitstreamStrided.c b/tests/src/inline/testBitstreamStrided.c index 3a946e139..7208684d1 100644 --- a/tests/src/inline/testBitstreamStrided.c +++ b/tests/src/inline/testBitstreamStrided.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField1d.c b/tests/src/misc/testZfpField1d.c index 745e550f4..7350c5a84 100644 --- a/tests/src/misc/testZfpField1d.c +++ b/tests/src/misc/testZfpField1d.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField1f.c b/tests/src/misc/testZfpField1f.c index 4a416cdfb..c56430a11 100644 --- a/tests/src/misc/testZfpField1f.c +++ b/tests/src/misc/testZfpField1f.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField2d.c b/tests/src/misc/testZfpField2d.c index 7c48af609..ae6313159 100644 --- a/tests/src/misc/testZfpField2d.c +++ b/tests/src/misc/testZfpField2d.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField2f.c b/tests/src/misc/testZfpField2f.c index ebba9fead..a302d34b7 100644 --- a/tests/src/misc/testZfpField2f.c +++ b/tests/src/misc/testZfpField2f.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField3d.c b/tests/src/misc/testZfpField3d.c index c084276be..c8a6fc674 100644 --- a/tests/src/misc/testZfpField3d.c +++ b/tests/src/misc/testZfpField3d.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField3f.c b/tests/src/misc/testZfpField3f.c index b4fca8265..caa78f1f5 100644 --- a/tests/src/misc/testZfpField3f.c +++ b/tests/src/misc/testZfpField3f.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField4d.c b/tests/src/misc/testZfpField4d.c index aeffbccb1..11c10bd99 100644 --- a/tests/src/misc/testZfpField4d.c +++ b/tests/src/misc/testZfpField4d.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpField4f.c b/tests/src/misc/testZfpField4f.c index 9fdd60396..87f7dc30e 100644 --- a/tests/src/misc/testZfpField4f.c +++ b/tests/src/misc/testZfpField4f.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include diff --git a/tests/src/misc/testZfpHeader.c b/tests/src/misc/testZfpHeader.c index 3986a739b..17904be39 100644 --- a/tests/src/misc/testZfpHeader.c +++ b/tests/src/misc/testZfpHeader.c @@ -3,6 +3,7 @@ #include #include +#include #include #include diff --git a/tests/src/misc/testZfpPromote.c b/tests/src/misc/testZfpPromote.c index 6416abee1..6f40c45a6 100644 --- a/tests/src/misc/testZfpPromote.c +++ b/tests/src/misc/testZfpPromote.c @@ -2,6 +2,7 @@ #include #include +#include #include #include diff --git a/tests/src/misc/testZfpStream.c b/tests/src/misc/testZfpStream.c index 4f885a2ea..3c5a0744c 100644 --- a/tests/src/misc/testZfpStream.c +++ b/tests/src/misc/testZfpStream.c @@ -2,6 +2,7 @@ #include #include +#include #include #include From a08dc81faf56757fff3deaf62e56e8919e4750dc Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 6 Oct 2025 16:11:33 -0700 Subject: [PATCH 253/277] Add missing stdint.h inclusions for cmocka --- tests/cfp/testCfpNamespace.c | 1 + tests/src/decode/zfpDecodeBlockStridedBase.c | 1 + tests/src/encode/zfpEncodeBlockStridedBase.c | 1 + 3 files changed, 3 insertions(+) diff --git a/tests/cfp/testCfpNamespace.c b/tests/cfp/testCfpNamespace.c index 87ac687fd..fe638bbf1 100644 --- a/tests/cfp/testCfpNamespace.c +++ b/tests/cfp/testCfpNamespace.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/decode/zfpDecodeBlockStridedBase.c b/tests/src/decode/zfpDecodeBlockStridedBase.c index cf921225b..0a1151c8a 100644 --- a/tests/src/decode/zfpDecodeBlockStridedBase.c +++ b/tests/src/decode/zfpDecodeBlockStridedBase.c @@ -1,5 +1,6 @@ #include #include +#include #include #include diff --git a/tests/src/encode/zfpEncodeBlockStridedBase.c b/tests/src/encode/zfpEncodeBlockStridedBase.c index 10294ac21..2e332fc1d 100644 --- a/tests/src/encode/zfpEncodeBlockStridedBase.c +++ b/tests/src/encode/zfpEncodeBlockStridedBase.c @@ -1,5 +1,6 @@ #include #include +#include #include #include From b2065319d9a662d4a63ee83f823ef35d53f4a7b8 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 8 Oct 2025 11:20:05 -0700 Subject: [PATCH 254/277] Add more missing stdint.h inclusions for cmocka --- tests/array/zfp/testAlignedMemory.cpp | 2 +- tests/cfp/testCfpArray_source.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/array/zfp/testAlignedMemory.cpp b/tests/array/zfp/testAlignedMemory.cpp index 0c94a5d21..07d34c1e4 100644 --- a/tests/array/zfp/testAlignedMemory.cpp +++ b/tests/array/zfp/testAlignedMemory.cpp @@ -6,7 +6,7 @@ using namespace zfp; #include "../utils/gtestSingleFixture.h" #include "../utils/predicates.h" -#include +#include TestEnv* const testEnv = new TestEnv; diff --git a/tests/cfp/testCfpArray_source.c b/tests/cfp/testCfpArray_source.c index de0983412..6859350ce 100644 --- a/tests/cfp/testCfpArray_source.c +++ b/tests/cfp/testCfpArray_source.c @@ -1,5 +1,6 @@ #include #include +#include #include #include From aa838835d55026de8cc0363766c6951f2f8f2545 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 8 Oct 2025 11:22:09 -0700 Subject: [PATCH 255/277] Discontinue Visual Studio 2015 testing incompatible with C++17 --- appveyor.yml | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/appveyor.yml b/appveyor.yml index 8a1e4688b..661fd1a37 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -17,18 +17,6 @@ environment: BUILD_TYPE: Release PYTHON_VERSION: 38 - - COMPILER: msvc - GENERATOR: Visual Studio 14 2015 Win64 - PLATFORM: x64 - BUILD_TYPE: Release - PYTHON_VERSION: 38 - - - COMPILER: msvc - GENERATOR: Visual Studio 14 2015 - PLATFORM: Win32 - BUILD_TYPE: Release - PYTHON_VERSION: 38 - - COMPILER: mingw GENERATOR: MinGW Makefiles PLATFORM: Win32 From d52f17452e05248ae1c781c16677bd859c8d95eb Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 8 Oct 2025 11:22:46 -0700 Subject: [PATCH 256/277] Migrate CUDA tests from Lassen to Matrix --- tests/gitlab/gitlab-ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/gitlab/gitlab-ci.yml b/tests/gitlab/gitlab-ci.yml index 1912f5ce5..3a7b92ac4 100644 --- a/tests/gitlab/gitlab-ci.yml +++ b/tests/gitlab/gitlab-ci.yml @@ -141,8 +141,8 @@ include: # - local: tests/gitlab/pascal-jobs.yml # - local: tests/gitlab/lassen-templates.yml # - local: tests/gitlab/lassen-jobs.yml -# - local: tests/gitlab/matrix-templates.yml -# - local: tests/gitlab/matrix-jobs.yml + - local: tests/gitlab/matrix-templates.yml + - local: tests/gitlab/matrix-jobs.yml - local: tests/gitlab/dane-templates.yml - local: tests/gitlab/dane-jobs.yml # - local: tests/gitlab/quartz-templates.yml From b9442d10e4c035c447e094767b50fda958a1319e Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 8 Oct 2025 14:05:48 -0700 Subject: [PATCH 257/277] Bump gcc to 10.3.1 on Matrix --- tests/gitlab/matrix-jobs.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/gitlab/matrix-jobs.yml b/tests/gitlab/matrix-jobs.yml index 535c9197d..e5edf17f4 100644 --- a/tests/gitlab/matrix-jobs.yml +++ b/tests/gitlab/matrix-jobs.yml @@ -6,7 +6,7 @@ cuda-11.8.0_build: variables: ci_cmake: "cmake/3.23.1" ci_cmp_mod: "cuda/11.8.0" - ci_gcc_mod: "gcc/8.3.1" + ci_gcc_mod: "gcc/10.3.1" extends: [.cuda, .matrix_build_gpu] needs: [] From 2432e82c8e8267ac1469d27a635f52284cce7598 Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 1 Dec 2025 12:02:30 -0800 Subject: [PATCH 258/277] Bump compiler to gcc-13 on macos-14 --- .github/workflows/tests.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index e3036acd6..39ab21235 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -44,8 +44,8 @@ jobs: # - os: macos-latest - os: macos-14 - cxx_compiler: g++-12 - c_compiler: gcc-12 + cxx_compiler: g++-13 + c_compiler: gcc-13 omp: ON target: all From cdeb417d3991417f7eff1db260a923b144a96fdd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 1 Dec 2025 21:06:27 +0000 Subject: [PATCH 259/277] Bump actions/checkout from 5 to 6 Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 92a00d04a..33387c4c6 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -11,7 +11,7 @@ jobs: if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: - name: Checkout - uses: actions/checkout@v5 + uses: actions/checkout@v6 - name: Setup Python uses: actions/setup-python@v6 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 093d9b870..08e0d2f7c 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -7,7 +7,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout Zfp - uses: actions/checkout@v5 + uses: actions/checkout@v6 - name: Setup Python uses: actions/setup-python@v6 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index 7b955ebc1..2395fe5a8 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -7,7 +7,7 @@ jobs: runs-on: macos-latest steps: - name: Checkout Zfp - uses: actions/checkout@v5 + uses: actions/checkout@v6 - name: Setup Python uses: actions/setup-python@v6 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 39ab21235..bebff0961 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -56,7 +56,7 @@ jobs: target: all steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@v6 - name: Setup Python uses: actions/setup-python@v6 From a6e386b486e594076fe27497edca17c0f3a162f9 Mon Sep 17 00:00:00 2001 From: William Allen <16820599+williamjallen@users.noreply.github.com> Date: Fri, 12 Dec 2025 13:50:14 -0500 Subject: [PATCH 260/277] Pin GHA dependencies by hash Hash-pinned dependencies are an important requirement enforced by the OpenSSF scorecard tool: https://github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies This commit pins all GitHub Actions workflows to a commit hash. --- .github/workflows/coverage.yml | 6 +++--- .github/workflows/debug-linux.yml | 6 +++--- .github/workflows/debug-macos.yml | 6 +++--- .github/workflows/tests.yml | 4 ++-- 4 files changed, 11 insertions(+), 11 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 33387c4c6..47f6d34bf 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -11,10 +11,10 @@ jobs: if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@v6 + uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 with: python-version: '3.x' architecture: x64 @@ -42,7 +42,7 @@ jobs: lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml - name: Upload Report to Codecov - uses: codecov/codecov-action@v5 + uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5 with: files: ${{github.workspace}}/build/coverage.xml env_vars: Actions diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 08e0d2f7c..7b1855c8a 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -7,10 +7,10 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout Zfp - uses: actions/checkout@v6 + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@v6 + uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 with: python-version: '3.x' architecture: x64 @@ -26,4 +26,4 @@ jobs: sudo apt-get update; sudo apt-get install -y libomp5 libomp-dev - name: Setup Tmate Session - uses: mxschmitt/action-tmate@v3 + uses: mxschmitt/action-tmate@c0afd6f790e3a5564914980036ebf83216678101 # v3 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index 2395fe5a8..e4dfa98f8 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -7,10 +7,10 @@ jobs: runs-on: macos-latest steps: - name: Checkout Zfp - uses: actions/checkout@v6 + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@v6 + uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 with: python-version: '3.x' architecture: x64 @@ -26,4 +26,4 @@ jobs: brew install libomp - name: Setup Tmate Session - uses: mxschmitt/action-tmate@v3 + uses: mxschmitt/action-tmate@c0afd6f790e3a5564914980036ebf83216678101 # v3 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index bebff0961..60ab972ed 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -56,10 +56,10 @@ jobs: target: all steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@v6 + uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 with: python-version: '3.x' architecture: ${{ matrix.architecture || 'x64' }} From 980fe65567694fc729dfa3b3f3d08fe6743e21fb Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 29 Dec 2025 13:57:19 -0800 Subject: [PATCH 261/277] Handle deprecated newshape parameter in np.reshape --- tests/python/test_numpy.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/python/test_numpy.py b/tests/python/test_numpy.py index e35ed043d..4f822c916 100644 --- a/tests/python/test_numpy.py +++ b/tests/python/test_numpy.py @@ -34,7 +34,12 @@ def test_different_dtypes(self): for dtype in [np.float32, np.float64]: elements = np.random.random_sample(num_elements) elements = elements.astype(dtype, casting="same_kind") - array = np.reshape(elements, newshape=shape) + if (version_parse is not None and + (version_parse(np.__version__) >= version_parse("2.1.0")) + ): + array = np.reshape(elements, shape=shape) + else: + array = np.reshape(elements, newshape=shape) self.lossless_round_trip(array) if (version_parse is not None and From 7eba67d56ad3d7544b328ec57f17dd9f091d088b Mon Sep 17 00:00:00 2001 From: lindstro Date: Mon, 29 Dec 2025 14:59:53 -0800 Subject: [PATCH 262/277] Install packaging for parsing Python package versions --- .github/workflows/tests.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 60ab972ed..536611c66 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -69,6 +69,7 @@ jobs: python -m pip install cython python -m pip install oldest-supported-numpy python -m pip install setuptools + python -m pip install packaging - name: Setup OpenMP (Linux) if: ${{(matrix.os == 'ubuntu-latest' || matrix.os == 'ubuntu-24.04-arm') && matrix.cxx_compiler == 'clang++'}} From 15360cededfe1711428676e1b99ae8b04c87b170 Mon Sep 17 00:00:00 2001 From: Mark Harfouche Date: Tue, 23 Dec 2025 21:00:04 -0500 Subject: [PATCH 263/277] Add a python version identifier Closes https://github.com/llnl/zfp/issues/280 --- CHANGELOG.md | 1 + python/zfpy.pxd | 1 + python/zfpy.pyx | 2 ++ tests/python/test_numpy.py | 5 +++++ 4 files changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7903a4210..7a6089b07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Change Log - A new code example, `chunk`, shows how to perform (de)compression in chunks. - A new utility function `zfp_block_maximum_size()` returns maximum block size for given scalar type, dimensionality, and compression mode. +- `zfpy.__version__` for straightfoward access to the zfp version string in Python. ### Fixed diff --git a/python/zfpy.pxd b/python/zfpy.pxd index 067bd90a4..87aea2cf3 100644 --- a/python/zfpy.pxd +++ b/python/zfpy.pxd @@ -11,6 +11,7 @@ cdef extern from "zfp/bitstream.h": void stream_close(bitstream* stream) cdef extern from "zfp.h": + cython.char * ZFP_VERSION_STRING # enums ctypedef enum zfp_type: zfp_type_none = 0, diff --git a/python/zfpy.pyx b/python/zfpy.pyx index 0d1eb5230..89b455cfc 100644 --- a/python/zfpy.pyx +++ b/python/zfpy.pyx @@ -37,6 +37,8 @@ mode_fixed_rate = zfp_mode_fixed_rate mode_fixed_precision = zfp_mode_fixed_precision mode_fixed_accuracy = zfp_mode_fixed_accuracy +__version__ = str(ZFP_VERSION_STRING, encoding='utf-8') + cpdef dtype_to_ztype(dtype): if dtype == np.int32: diff --git a/tests/python/test_numpy.py b/tests/python/test_numpy.py index 4f822c916..189aa430b 100644 --- a/tests/python/test_numpy.py +++ b/tests/python/test_numpy.py @@ -10,6 +10,11 @@ except ImportError: version_parse = None +def test_zfpy_version(): + # Just ensure that the version contains 3 numbers separated by dots + assert len(zfpy.__version__.split('.')) == 3 + # Ensure it is a string, not bytes + assert isinstance(zfpy.__version__, str) class TestNumpy(unittest.TestCase): def lossless_round_trip(self, orig_array): From f29f730de86382a2501958d8f301db6f407343a5 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 30 Dec 2025 08:42:42 -0800 Subject: [PATCH 264/277] Correct description of ZFP_VERSION_STRING --- docs/source/high-level-api.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/source/high-level-api.rst b/docs/source/high-level-api.rst index 1d13c0df6..f3f59c425 100644 --- a/docs/source/high-level-api.rst +++ b/docs/source/high-level-api.rst @@ -84,9 +84,11 @@ Macros .. c:macro:: ZFP_VERSION_STRING - :c:macro:`ZFP_VERSION_STRING` is a string literal composed of the four - :ref:`version identifiers `. It is a component of - :c:data:`zfp_version_string`. + :c:macro:`ZFP_VERSION_STRING` is a string literal composed of the three + to four :ref:`version identifiers `. The string does not + include the fourth identifier, :c:macro:`ZFP_VERSION_TWEAK`, if it is + zero. For example, version `1.2.3.0` is identified as `"1.2.3"`. This + macro is one of the components of :c:data:`zfp_version_string`. ---- From f58d1d57873bb7ddc16f3cfd93200d384deb3c61 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 30 Dec 2025 08:43:09 -0800 Subject: [PATCH 265/277] Document zfpy.__version__ --- docs/source/defs.rst | 1 + docs/source/python.rst | 13 +++++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/source/defs.rst b/docs/source/defs.rst index 4e29ac485..f78886edc 100644 --- a/docs/source/defs.rst +++ b/docs/source/defs.rst @@ -41,3 +41,4 @@ .. |cpprelease| replace:: 1.0.0 .. |verrelease| replace:: 1.0.0 .. |roundingrelease| replace:: 1.0.0 +.. |nextrelease| replace:: 1.1.0 diff --git a/docs/source/python.rst b/docs/source/python.rst index 92373c814..48dbd2d65 100644 --- a/docs/source/python.rst +++ b/docs/source/python.rst @@ -13,8 +13,17 @@ floating-point arrays. The |zfpy| implementation is based on `Cython `_ and requires both NumPy and Cython to be installed. Currently, |zfpy| supports only serial execution. -The |zfpy| API is limited to two functions, for compression and -decompression, which are described below. +The |zfpy| API is limited to two functions, for compression and decompression, +and a version identifier, which are described below. + +Constants +--------- + +.. py:data:: __version__ + + Python string identical to :c:macro:`ZFP_VERSION_STRING` representing the + |zfpy| library version, e.g., `'1.1.0'`. Available as of |zfp| + |nextrelease|. Compression ----------- From 52a2f277efba1ba1be8e8022e3744e9a13425dca Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 30 Dec 2025 08:45:11 -0800 Subject: [PATCH 266/277] Bump compiler to gcc-14 for codecov --- .github/workflows/coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 47f6d34bf..d4de11784 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -25,7 +25,7 @@ jobs: python -m pip install lcov_cobertura - name: Run CMake - run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-11 -DCMAKE_C_COMPILER=gcc-11 -DCMAKE_CXX_FLAGS="-fprofile-arcs -ftest-coverage" -DCMAKE_C_FLAGS="-fprofile-arcs -ftest-coverage" -DBUILD_TESTING_FULL=ON -DBUILD_CFP=ON -DZFP_WITH_OPENMP=ON + run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-14 -DCMAKE_C_COMPILER=gcc-14 -DCMAKE_CXX_FLAGS="-fprofile-arcs -ftest-coverage" -DCMAKE_C_FLAGS="-fprofile-arcs -ftest-coverage" -DBUILD_TESTING_FULL=ON -DBUILD_CFP=ON -DZFP_WITH_OPENMP=ON - name: Build run: cmake --build ${{github.workspace}}/build From a2c8eff19ab70cfee89c1dc49a3734442414e753 Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 30 Dec 2025 09:22:29 -0800 Subject: [PATCH 267/277] Revert to gcc-13 for gcov compatibility --- .github/workflows/coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index d4de11784..aeb1ccef0 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -25,7 +25,7 @@ jobs: python -m pip install lcov_cobertura - name: Run CMake - run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-14 -DCMAKE_C_COMPILER=gcc-14 -DCMAKE_CXX_FLAGS="-fprofile-arcs -ftest-coverage" -DCMAKE_C_FLAGS="-fprofile-arcs -ftest-coverage" -DBUILD_TESTING_FULL=ON -DBUILD_CFP=ON -DZFP_WITH_OPENMP=ON + run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-13 -DCMAKE_C_COMPILER=gcc-13 -DCMAKE_CXX_FLAGS="-fprofile-arcs -ftest-coverage" -DCMAKE_C_FLAGS="-fprofile-arcs -ftest-coverage" -DBUILD_TESTING_FULL=ON -DBUILD_CFP=ON -DZFP_WITH_OPENMP=ON - name: Build run: cmake --build ${{github.workspace}}/build From 920172e347249e02ce82f320e866b52ba5eb5f23 Mon Sep 17 00:00:00 2001 From: lindstro Date: Wed, 31 Dec 2025 16:12:22 -0800 Subject: [PATCH 268/277] Suppress lcov mismatch error --- .github/workflows/coverage.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index aeb1ccef0..220a58fe1 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -36,8 +36,9 @@ jobs: - name: Generate Coverage Report working-directory: ${{github.workspace}}/build + # for now, suppress mismatch errors (likely gcc/gcov bug) run: |- - lcov -c --directory ${{github.workspace}}/build --output-file coverage.info + lcov -c --directory ${{github.workspace}}/build --output-file coverage.info --ignore-errors mismatch lcov --remove coverage.info '${{github.workspace}}/build/tests/*' --remove coverage.info '${{github.workspace}}/tests/*' --remove coverage.info '/usr/include/*' -o coverage.info lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml From 132f1cb6f305e241080cb9859d9055c589fb6e2e Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 1 Jan 2026 12:00:24 -0800 Subject: [PATCH 269/277] Add Codecov token --- .github/workflows/coverage.yml | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 220a58fe1..8ea7aa859 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -44,7 +44,9 @@ jobs: - name: Upload Report to Codecov uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5 + env: + CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} with: - files: ${{github.workspace}}/build/coverage.xml - env_vars: Actions - fail_ci_if_error: true + files: ${{github.workspace}}/build/coverage.xml + env_vars: Actions + fail_ci_if_error: true From a888ab7606c58cc5ab5f34298fec25473e0d28ad Mon Sep 17 00:00:00 2001 From: lindstro Date: Thu, 1 Jan 2026 13:14:25 -0800 Subject: [PATCH 270/277] Set Codecov token via token: line --- .github/workflows/coverage.yml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 8ea7aa859..ff5979ba9 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -44,9 +44,8 @@ jobs: - name: Upload Report to Codecov uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5 - env: - CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} with: files: ${{github.workspace}}/build/coverage.xml - env_vars: Actions + token: ${{secrets.CODECOV_TOKEN}} fail_ci_if_error: true + verbose: true From d626125a8f70a9adcb9eed20a18230efab2c9186 Mon Sep 17 00:00:00 2001 From: William Allen Date: Sun, 25 Jan 2026 12:19:45 -0500 Subject: [PATCH 271/277] Restrict workflow permissions to minimum required Limiting the scope of the GITHUB_TOKEN via the [permissions](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#permissions) attribute is best practice to prevent a leaked GITHUB_TOKEN from allowing repository write access. --- .github/workflows/coverage.yml | 3 +++ .github/workflows/debug-linux.yml | 3 +++ .github/workflows/debug-macos.yml | 3 +++ .github/workflows/tests.yml | 3 +++ 4 files changed, 12 insertions(+) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index ff5979ba9..c6cfc6c69 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -5,6 +5,9 @@ on: workflows: [Tests] types: [completed] +permissions: + contents: read + jobs: coverage: runs-on: ubuntu-latest diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 7b1855c8a..309efa5e0 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -2,6 +2,9 @@ name: Debug (Linux) on: [workflow_dispatch] +permissions: + contents: read + jobs: debug: runs-on: ubuntu-latest diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index e4dfa98f8..bc015bd9c 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -2,6 +2,9 @@ name: Debug (MacOS) on: [workflow_dispatch] +permissions: + contents: read + jobs: debug: runs-on: macos-latest diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 536611c66..6012c7b65 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -2,6 +2,9 @@ name: Tests on: [push, pull_request] +permissions: + contents: read + env: BUILD_TYPE: Release From 732e9ff85011f49dad8adfbc4476d65a77caa885 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 26 Jan 2026 14:33:56 +0000 Subject: [PATCH 272/277] Bump actions/setup-python from 6.1.0 to 6.2.0 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.1.0 to 6.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/83679a892e2d95755f2dac6acb0bfd1e9ac5d548...a309ff8b426b58ec0e2a45f0f869d46889d02405) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index c6cfc6c69..ce397cecc 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -17,7 +17,7 @@ jobs: uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 + uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 309efa5e0..33bc21415 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -13,7 +13,7 @@ jobs: uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 + uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index bc015bd9c..181613eca 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -13,7 +13,7 @@ jobs: uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 + uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 with: python-version: '3.x' architecture: x64 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 6012c7b65..2858c70f8 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -62,7 +62,7 @@ jobs: - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 - name: Setup Python - uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6 + uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 with: python-version: '3.x' architecture: ${{ matrix.architecture || 'x64' }} From 39922a1a6282f1b9290edc175aca4556f5148195 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 26 Jan 2026 18:48:43 +0000 Subject: [PATCH 273/277] Bump actions/checkout from 6.0.1 to 6.0.2 Bumps [actions/checkout](https://github.com/actions/checkout) from 6.0.1 to 6.0.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/8e8c483db84b4bee98b60c0593521ed34d9990e8...de0fac2e4500dabe0009e67214ff5f5447ce83dd) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- .github/workflows/debug-linux.yml | 2 +- .github/workflows/debug-macos.yml | 2 +- .github/workflows/tests.yml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index ce397cecc..1e435dd41 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -14,7 +14,7 @@ jobs: if: ${{ github.event.workflow_run.conclusion == 'success' }} steps: - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - name: Setup Python uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 diff --git a/.github/workflows/debug-linux.yml b/.github/workflows/debug-linux.yml index 33bc21415..de77f45c7 100644 --- a/.github/workflows/debug-linux.yml +++ b/.github/workflows/debug-linux.yml @@ -10,7 +10,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout Zfp - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - name: Setup Python uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 diff --git a/.github/workflows/debug-macos.yml b/.github/workflows/debug-macos.yml index 181613eca..2197f68d2 100644 --- a/.github/workflows/debug-macos.yml +++ b/.github/workflows/debug-macos.yml @@ -10,7 +10,7 @@ jobs: runs-on: macos-latest steps: - name: Checkout Zfp - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - name: Setup Python uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 2858c70f8..1494c0e35 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -59,7 +59,7 @@ jobs: target: all steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - name: Setup Python uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6 From 4d6163105851f3abe5f3d4320b6be77134d5ebab Mon Sep 17 00:00:00 2001 From: lindstro Date: Tue, 27 Jan 2026 07:44:00 -0800 Subject: [PATCH 274/277] Suppress NumPy deprecated API warnings --- setup.py | 1 + 1 file changed, 1 insertion(+) diff --git a/setup.py b/setup.py index b4bc530ec..0661368dc 100644 --- a/setup.py +++ b/setup.py @@ -22,6 +22,7 @@ def __repr__(self): Extension( "zfpy", sources=["python/zfpy.pyx"], + define_macros=[('NPY_NO_DEPRECATED_API', 'NPY_1_7_API_VERSION')], include_dirs=["include", str(NumpyImport())], libraries=["zfp"], library_dirs=["build/lib64", "build/lib/Release"], From c4cba153779268b2930585349ff8ed356c9cc83c Mon Sep 17 00:00:00 2001 From: lindstro Date: Fri, 30 Jan 2026 13:58:30 -0800 Subject: [PATCH 275/277] Add security policy --- SECURITY.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 SECURITY.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000000000..e92d7de6d --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,19 @@ +# Security Policy + +The zfp team takes software security and safety issues seriously. We will +work to resolve security issues in a timely manner as they become known. + +## Supported Versions + +Security updates to zfp are generally applied only to the latest release. + +## Reporting a Vulnerability + +If you have discovered a zfp vulnerability, please report it privately. +**Do not disclose it as a public issue.** +This gives us time to work with you to fix the issue before public exposure, +reducing the chance that the vulnerability will be exploited before a patch +is released. + +Please [report security issues](https://github.com/LLNL/zfp/security/advisories/new) +on the GitHub Security tab. From 93649399ced410f811c97dc949b49cdb78d877e3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 23 Mar 2026 13:33:05 +0000 Subject: [PATCH 276/277] Bump codecov/codecov-action from 5.5.2 to 5.5.3 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.5.2 to 5.5.3. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/671740ac38dd9b0130fbe1cec585b89eea48d3de...1af58845a975a7985b0beb0cbe6fbbb71a41dbad) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-version: 5.5.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index 1e435dd41..e96e9b6e8 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -46,7 +46,7 @@ jobs: lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml - name: Upload Report to Codecov - uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5 + uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5 with: files: ${{github.workspace}}/build/coverage.xml token: ${{secrets.CODECOV_TOKEN}} From 93414908af924949b0e61662cfc0daa6966848c2 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 30 Mar 2026 14:32:27 +0000 Subject: [PATCH 277/277] Bump codecov/codecov-action from 5.5.3 to 6.0.0 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.5.3 to 6.0.0. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/codecov/codecov-action/compare/1af58845a975a7985b0beb0cbe6fbbb71a41dbad...57e3a136b779b570ffcdbf80b3bdc90e7fab3de2) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] --- .github/workflows/coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml index e96e9b6e8..69c553908 100644 --- a/.github/workflows/coverage.yml +++ b/.github/workflows/coverage.yml @@ -46,7 +46,7 @@ jobs: lcov_cobertura ${{github.workspace}}/build/coverage.info -d -o ${{github.workspace}}/build/coverage.xml - name: Upload Report to Codecov - uses: codecov/codecov-action@1af58845a975a7985b0beb0cbe6fbbb71a41dbad # v5 + uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v5 with: files: ${{github.workspace}}/build/coverage.xml token: ${{secrets.CODECOV_TOKEN}}