Copyright 2000, 2001, 2002 Free Software Foundation, Inc. This file is part of the GNU MP Library. The GNU MP Library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. The GNU MP Library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with the GNU MP Library; see the file COPYING.LIB. If not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. The IA-64 ISA keeps instructions three and three in 128 bit bundles. Programmers/compilers need to put explicit breaks `;;' when there are WAW or RAW dependencies. Such breaks can typically just be at the end of a bundle, with some exceptions. The Itanium and Itanium 2 implementations can under ideal conditions execute two bundles per cycle. The Itanium 2 allows 4 of these instructions to do integer operations, while the Itanium 2 allows all 6 to be integer operations. Taken cloop branches seem to insert a bubble into the pipeline most of the time. Loads to the fp registers bypass the L1 cache and thus get extremely long latencies, 9 cycles on the Itanium and 7 cycles on the Itanium 2. The software pipeline stuff using br.ctop instruction causes delays, since many issue slots are taken up by instructions with zero predicates, and since about many extra instructions are needed to set things up. These features are designed for code density, not maximum speed. Misc pipeline limitations (Itanium): * The getf.sig instruction can only execute in M0. * At most four integer instructions/cycle. * Nops take up resources like any plain instructions. ================================================================ mpn_add_n, mpn_sub_n: The current code runs at 3 cycles/limb. Unrolling could clearly bring down the time to 2 cycles/limb. ================================================================ mpn_addmul_1: The current code runs at 3.7 cycles/limb, but that somewhat odd timing is reached only for huge operands. It uses the mod-scheduled software pipelining feature. The reason for the poor speed for small operands is that mod-scheduled loops have a very long start-up overhead. For optimal speed, we need to load two 64-bit limbs with the ldfp8 instruction, and stay away from mod-scheduled loops. Since rp and up might be mutually aligned in two ways, we will need two loop variants, with the same basic structure: { .mfi getf.sig xma.l (p6) cmp.leu p6, p7 = } { .mfi stf8 xma.hu (p7) cmp.ltu p6, p7 = ;; } { .mib getf.sig (p6) add 1 nop.b } { .mib ldfp8 = [up], 16 (p7) add nop.b ;; { .mfi getf.sig xma.l (p6) cmp.leu p6, p7 = } { .mfi stf8 xma.hu (p7) cmp.ltu p6, p7 = ;; } { .mib getf.sig (p6) add 1 nop.b } { .mib ldfp8 = [rp], 16 (p7) add br.cloop ;; } 2 limbs/20 instructions 20 insn/max 6 insn/cycle: 3.3 cycles/2limb 8 memops/max 2 memops/cycle: 4.0 cycles/2limb 8 intops/max 2 intops/cycle: 4.0 cycles/2limb 4 fpops/max 2 fpops/cycle: 2.0 cycles/2limb ================================================================ mpn_submul_1: The current code just calls mpn_mul_1 and mpn_sub_n and thus needs about 7 cycles/limb. We could implement this much like mpn_addmul_1, if we first complement v. When v is complemented, the low product limb becomes complement of true product. This should allow us to use the accumulation of xma. Here is how it works: #define umul_ppmma(ph, pl, m0, m1, a) \ do { \ UDItype __m0 = (m0), __m1 = (m1), __a = (a); \ __asm__ ("xma.hu %0 = %1, %2, %3" \ : "=f" (ph) \ : "f" (m0), "f" (m1), "f" (__a)); \ (pl) = __m0 * __m1 + __a; \ } while (0) mp_limb_t mpn_submul_1 (mp_ptr rp, mp_srcptr up, mp_size_t n, mp_limb_t vl) { mp_limb_t cl, cy; mp_size_t i; mp_limb_t phi, plo; mp_limb_t x; mp_limb_t ul, vln; vln = -vl; cl = 0; for (i = n; i != 0; i--) { ul = *up++; /* will need this in both fregs and gregs */ x = *rp; umul_ppmma (phi, plo, ul, vln, x); cy = plo < cl; plo -= cl; cl = ul - phi; cl += cy; *rp++ = plo; } return cl; } ================================================================ mpn_mul_1: The current code runs at 3.7 cycles/limb. The code is very similar to the mpn_addmul_1 code. See comments above. Faster code wouldn't be too hard to write. This is one possible pattern: { .mfi getf.sig xma.l (p6) cmp.leu p6, p7 = } { .mfi stf8 xma.hu (p7) cmp.ltu p6, p7 = ;; } { .mib getf.sig (p6) add 1 nop.b } { .mib ldf8 = [up], 8 (p7) add br.cloop ;; } 1 limb/12 instructions 12 insn/max 6 insn/cycle: 2.0 cycles/limb 4 memops/max 2 memops/cycle: 2.0 cycles/limb 4 intops/max 2 intops/cycle: 2.0 cycles/limb 2 fpops/max 2 fpops/cycle: 1.0 cycles/limb ================================================================ mpn_mul_8 The add+cmp+add we use on the other codes is optimal for shortening recurrencies (2 cycles) but the sequence takes up 4 execution slots. When recurrency depth is not critical, a more standard 3-cycle add+cmp+add is better. /* First load the 8 values from v */ ldfp8 v0, v1 = [r35], 16;; ldfp8 v2, v3 = [r35], 16;; ldfp8 v4, v5 = [r35], 16;; ldfp8 v6, v7 = [r35], 16;; /* In the inner loop, get a new U limb and store a result limb. */ mov lc = un Loop: ldf8 u0 = [r33], 8 xma.l lp0 = v0, u0, hp0 xma.hu hp0 = v0, u0, hp0 xma.l lp1 = v1, u0, hp1 xma.hu hp1 = v1, u0, hp1 xma.l lp2 = v2, u0, hp2 xma.hu hp2 = v2, u0, hp2 xma.l lp3 = v3, u0, hp3 xma.hu hp3 = v3, u0, hp3 xma.l lp4 = v4, u0, hp4 xma.hu hp4 = v4, u0, hp4 xma.l lp5 = v5, u0, hp5 xma.hu hp5 = v5, u0, hp5 xma.l lp6 = v6, u0, hp6 xma.hu hp6 = v6, u0, hp6 xma.l lp7 = v7, u0, hp7 xma.hu hp7 = v7, u0, hp7 getf.sig l0 = lp0 getf.sig l1 = lp1 getf.sig l2 = lp2 getf.sig l3 = lp3 getf.sig l4 = lp4 getf.sig l5 = lp5 getf.sig l6 = lp6 getf.sig l7 = lp7 add+cmp+add l0, l0, h0 add+cmp+add l1, l1, h1 add+cmp+add l2, l2, h2 add+cmp+add l3, l3, h3 add+cmp+add l4, l4, h4 add+cmp+add l5, l5, h5 add+cmp+add l6, l6, h6 add+cmp+add l7, l7, h7 st8 [r32] = xx, 8 br.cloop Loop 50 insn at max 6 insn/cycle: 8.33 cycles/limb8 10 memops at max 2 memops/cycle: 5 cycles/limb8 16 fpops at max 2 fpops/cycle: 8 cycles/limb8 24 intops at max 4 intops/cycle: 6 cycles/limb8 10+24 memops+intops at max 4/cycle 8.5 cycles/limb8 1.0625 cycles/limb ================================================================ mpn_lshift, mpn_rshift The current code runs at 2 cycles/limb, but has a too deep software pipeline. The code suffers badly from the 4 cycle latency of the variable shift instructions. Using 63 separate loops, we could use the double-word SHRP instruction. That instruction has a plain single-cycle latency. We need 63 loops since this instruction only accept immediate count.