![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to submul_1.asm CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [local] / OpenXM_contrib / gmp / mpn / alpha / ev6|
Return to submul_1.asm CVS log | Up to [local] / OpenXM_contrib / gmp / mpn / alpha / ev6 |
1.1 ohara 1: dnl Alpha ev6 mpn_submul_1 -- Multiply a limb vector with a limb and subtract
2: dnl the result from a second limb vector.
3:
4: dnl Copyright 2000 Free Software Foundation, Inc.
5:
6: dnl This file is part of the GNU MP Library.
7:
8: dnl The GNU MP Library is free software; you can redistribute it and/or modify
9: dnl it under the terms of the GNU Lesser General Public License as published
10: dnl by the Free Software Foundation; either version 2.1 of the License, or (at
11: dnl your option) any later version.
12:
13: dnl The GNU MP Library is distributed in the hope that it will be useful, but
14: dnl WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
15: dnl or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
16: dnl License for more details.
17:
18: dnl You should have received a copy of the GNU Lesser General Public License
19: dnl along with the GNU MP Library; see the file COPYING.LIB. If not, write to
20: dnl the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
21: dnl MA 02111-1307, USA.
22:
23: include(`../config.m4')
24:
25: dnl INPUT PARAMETERS
26: dnl res_ptr r16
27: dnl s1_ptr r17
28: dnl size r18
29: dnl s2_limb r19
30:
31: dnl This code runs at 42 cycles/limb on EV4, 18 cycles/limb on EV5, and
32: dnl exactly 3.5 cycles/limb on EV6...
33:
34: dnl This code was written in close cooperation with ev6 pipeline expert
35: dnl Steve Root. Any errors are tege's fault, though.
36: dnl
37: dnl Register usages for unrolled loop:
38: dnl 0-3 mul's
39: dnl 4-7 acc's
40: dnl 8-15 mul results
41: dnl 20,21 carry's
42: dnl 22,23 save for stores
43:
44: dnl Sustains 8 mul-adds in 28 cycles in the unrolled inner loop.
45:
46: dnl The stores can issue a cycle late so we have paired no-op's to 'catch'
47: dnl them, so that further disturbance to the schedule is damped.
48:
49: dnl We couldn't pair the loads, because the entangled schedule of the
50: dnl carry's has to happen on one side {0} of the machine. Note, the total
51: dnl use of U0, and the total use of L0 (after attending to the stores).
52: dnl which is part of the reason why....
53:
54: dnl This is a great schedule for the d_cache, a poor schedule for the
55: dnl b_cache. The lockup on U0 means that any stall can't be recovered
56: dnl from. Consider a ldq in L1. say that load gets stalled because it
57: dnl collides with a fill from the b_Cache. On the next cycle, this load
58: dnl gets priority. If first looks at L0, and goes there. The instruction
59: dnl we intended for L0 gets to look at L1, which is NOT where we want
60: dnl it. It either stalls 1, because it can't go in L0, or goes there, and
61: dnl causes a further instruction to stall.
62:
63: dnl So for b_cache, we're likely going to want to put one or more cycles
64: dnl back into the code! And, of course, put in prefetches. For the
65: dnl accumulator, lds, intent to modify. For the multiplier, you might
66: dnl want ldq, evict next, if you're not wanting to use it again soon. Use
67: dnl 256 ahead of present pointer value. At a place where we have an mt
68: dnl followed by a bookkeeping, put the bookkeeping in upper, and the
69: dnl prefetch into lower.
70:
71: dnl Note, the usage of physical registers per cycle is smoothed off, as
72: dnl much as possible.
73:
74: dnl Note, the ldq's and stq's are at the end of the quadpacks. note, we'd
75: dnl like not to have a ldq or stq to preceded a conditional branch in a
76: dnl quadpack. The conditional branch moves the retire pointer one cycle
77: dnl later.
78:
79: dnl Optimization notes:
80: dnl Callee-saves regs: r9 r10 r11 r12 r13 r14 r15 r26 ?r27?
81: dnl Reserved regs: r29 r30 r31
82: dnl Free caller-saves regs in unrolled code: r24 r25 r28
83: dnl We should swap some of the callee-saves regs for some of the free
84: dnl caller-saves regs, saving some overhead cycles.
85: dnl Most importantly, we should write fast code for the 0-7 case.
86: dnl The code we use there are for the 21164, and runs at 7 cycles/limb
87: dnl on the 21264. Should not be hard, if we write specialized code for
88: dnl 1-7 limbs (the one for 0 limbs should be straightforward). We then just
89: dnl need a jump table indexed by the low 3 bits of the count argument.
90:
91:
92: ASM_START()
93: PROLOGUE(mpn_submul_1)
94: cmpult r18, 8, r1
95: beq r1, $Large
96:
97: ldq r2, 0(r17) C r2 = s1_limb
98: addq r17, 8, r17 C s1_ptr++
99: subq r18, 1, r18 C size--
100: mulq r2, r19, r3 C r3 = prod_low
101: ldq r5, 0(r16) C r5 = *res_ptr
102: umulh r2, r19, r0 C r0 = prod_high
103: beq r18, $Lend0b C jump if size was == 1
104: ldq r2, 0(r17) C r2 = s1_limb
105: addq r17, 8, r17 C s1_ptr++
106: subq r18, 1, r18 C size--
107: subq r5, r3, r3
108: cmpult r5, r3, r4
109: stq r3, 0(r16)
110: addq r16, 8, r16 C res_ptr++
111: beq r18, $Lend0a C jump if size was == 2
112:
113: ALIGN(8)
114: $Loop0: mulq r2, r19, r3 C r3 = prod_low
115: ldq r5, 0(r16) C r5 = *res_ptr
116: addq r4, r0, r0 C cy_limb = cy_limb + 'cy'
117: subq r18, 1, r18 C size--
118: umulh r2, r19, r4 C r4 = cy_limb
119: ldq r2, 0(r17) C r2 = s1_limb
120: addq r17, 8, r17 C s1_ptr++
121: addq r3, r0, r3 C r3 = cy_limb + prod_low
122: cmpult r3, r0, r0 C r0 = carry from (cy_limb + prod_low)
123: subq r5, r3, r3
124: cmpult r5, r3, r5
125: stq r3, 0(r16)
126: addq r16, 8, r16 C res_ptr++
127: addq r5, r0, r0 C combine carries
128: bne r18, $Loop0
129: $Lend0a:
130: mulq r2, r19, r3 C r3 = prod_low
131: ldq r5, 0(r16) C r5 = *res_ptr
132: addq r4, r0, r0 C cy_limb = cy_limb + 'cy'
133: umulh r2, r19, r4 C r4 = cy_limb
134: addq r3, r0, r3 C r3 = cy_limb + prod_low
135: cmpult r3, r0, r0 C r0 = carry from (cy_limb + prod_low)
136: subq r5, r3, r3
137: cmpult r5, r3, r5
138: stq r3, 0(r16)
139: addq r5, r0, r0 C combine carries
140: addq r4, r0, r0 C cy_limb = prod_high + cy
141: ret r31, (r26), 1
142: $Lend0b:
143: subq r5, r3, r3
144: cmpult r5, r3, r5
145: stq r3, 0(r16)
146: addq r0, r5, r0
147: ret r31, (r26), 1
148:
149: $Large:
150: lda $30, -240($30)
151: stq $9, 8($30)
152: stq $10, 16($30)
153: stq $11, 24($30)
154: stq $12, 32($30)
155: stq $13, 40($30)
156: stq $14, 48($30)
157: stq $15, 56($30)
158:
159: and r18, 7, r20 C count for the first loop, 0-7
160: srl r18, 3, r18 C count for unrolled loop
161: bis r31, r31, r0
162: beq r20, $Lunroll
163: ldq r2, 0(r17) C r2 = s1_limb
164: addq r17, 8, r17 C s1_ptr++
165: subq r20, 1, r20 C size--
166: mulq r2, r19, r3 C r3 = prod_low
167: ldq r5, 0(r16) C r5 = *res_ptr
168: umulh r2, r19, r0 C r0 = prod_high
169: beq r20, $Lend1b C jump if size was == 1
170: ldq r2, 0(r17) C r2 = s1_limb
171: addq r17, 8, r17 C s1_ptr++
172: subq r20, 1, r20 C size--
173: subq r5, r3, r3
174: cmpult r5, r3, r4
175: stq r3, 0(r16)
176: addq r16, 8, r16 C res_ptr++
177: beq r20, $Lend1a C jump if size was == 2
178:
179: ALIGN(8)
180: $Loop1: mulq r2, r19, r3 C r3 = prod_low
181: ldq r5, 0(r16) C r5 = *res_ptr
182: addq r4, r0, r0 C cy_limb = cy_limb + 'cy'
183: subq r20, 1, r20 C size--
184: umulh r2, r19, r4 C r4 = cy_limb
185: ldq r2, 0(r17) C r2 = s1_limb
186: addq r17, 8, r17 C s1_ptr++
187: addq r3, r0, r3 C r3 = cy_limb + prod_low
188: cmpult r3, r0, r0 C r0 = carry from (cy_limb + prod_low)
189: subq r5, r3, r3
190: cmpult r5, r3, r5
191: stq r3, 0(r16)
192: addq r16, 8, r16 C res_ptr++
193: addq r5, r0, r0 C combine carries
194: bne r20, $Loop1
195:
196: $Lend1a:
197: mulq r2, r19, r3 C r3 = prod_low
198: ldq r5, 0(r16) C r5 = *res_ptr
199: addq r4, r0, r0 C cy_limb = cy_limb + 'cy'
200: umulh r2, r19, r4 C r4 = cy_limb
201: addq r3, r0, r3 C r3 = cy_limb + prod_low
202: cmpult r3, r0, r0 C r0 = carry from (cy_limb + prod_low)
203: subq r5, r3, r3
204: cmpult r5, r3, r5
205: stq r3, 0(r16)
206: addq r16, 8, r16 C res_ptr++
207: addq r5, r0, r0 C combine carries
208: addq r4, r0, r0 C cy_limb = prod_high + cy
209: br r31, $Lunroll
210: $Lend1b:
211: subq r5, r3, r3
212: cmpult r5, r3, r5
213: stq r3, 0(r16)
214: addq r16, 8, r16 C res_ptr++
215: addq r0, r5, r0
216:
217: $Lunroll:
218: lda r17, -16(r17) C L1 bookkeeping
219: lda r16, -16(r16) C L1 bookkeeping
220: bis r0, r31, r12
221:
222: C ____ UNROLLED LOOP SOFTWARE PIPELINE STARTUP ____
223:
224: ldq r2, 16(r17) C L1
225: ldq r3, 24(r17) C L1
226: lda r18, -1(r18) C L1 bookkeeping
227: ldq r6, 16(r16) C L1
228: ldq r7, 24(r16) C L1
229: ldq r0, 32(r17) C L1
230: mulq r19, r2, r13 C U1
231: ldq r1, 40(r17) C L1
232: umulh r19, r2, r14 C U1
233: mulq r19, r3, r15 C U1
234: lda r17, 64(r17) C L1 bookkeeping
235: ldq r4, 32(r16) C L1
236: ldq r5, 40(r16) C L1
237: umulh r19, r3, r8 C U1
238: ldq r2, -16(r17) C L1
239: mulq r19, r0, r9 C U1
240: ldq r3, -8(r17) C L1
241: umulh r19, r0, r10 C U1
242: subq r6, r13, r13 C L0 lo + acc
243: mulq r19, r1, r11 C U1
244: cmpult r6, r13, r20 C L0 lo add => carry
245: lda r16, 64(r16) C L1 bookkeeping
246: subq r13, r12, r22 C U0 hi add => answer
247: cmpult r13, r12, r21 C L0 hi add => carry
248: addq r14, r20, r14 C U0 hi mul + carry
249: ldq r6, -16(r16) C L1
250: subq r7, r15, r28 C L0 lo + acc
251: addq r14, r21, r14 C U0 hi mul + carry
252: cmpult r7, r15, r20 C L0 lo add => carry
253: ldq r7, -8(r16) C L1
254: umulh r19, r1, r12 C U1
255: subq r28, r14, r23 C U0 hi add => answer
256: ldq r0, 0(r17) C L1
257: mulq r19, r2, r13 C U1
258: cmpult r28, r14, r21 C L0 hi add => carry
259: addq r8, r20, r8 C U0 hi mul + carry
260: ldq r1, 8(r17) C L1
261: umulh r19, r2, r14 C U1
262: subq r4, r9, r9 C L0 lo + acc
263: stq r22, -48(r16) C L0
264: stq r23, -40(r16) C L1
265: mulq r19, r3, r15 C U1
266: addq r8, r21, r8 C U0 hi mul + carry
267: cmpult r4, r9, r20 C L0 lo add => carry
268: subq r9, r8, r22 C U0 hi add => answer
269: ble r18, $Lend C U1 bookkeeping
270:
271: C ____ MAIN UNROLLED LOOP ____
272: ALIGN(16)
273: $Loop:
274: bis r31, r31, r31 C U1 mt
275: cmpult r9, r8, r21 C L0 hi add => carry
276: addq r10, r20, r10 C U0 hi mul + carry
277: ldq r4, 0(r16) C L1
278:
279: bis r31, r31, r31 C U1 mt
280: subq r5, r11, r23 C L0 lo + acc
281: addq r10, r21, r10 C L0 hi mul + carry
282: ldq r2, 16(r17) C L1
283:
284: umulh r19, r3, r8 C U1
285: cmpult r5, r11, r20 C L0 lo add => carry
286: subq r23, r10, r28 C U0 hi add => answer
287: ldq r5, 8(r16) C L1
288:
289: mulq r19, r0, r9 C U1
290: cmpult r23, r10, r21 C L0 hi add => carry
291: addq r12, r20, r12 C U0 hi mul + carry
292: ldq r3, 24(r17) C L1
293:
294: umulh r19, r0, r10 C U1
295: subq r6, r13, r13 C L0 lo + acc
296: stq r22, -32(r16) C L0
297: stq r28, -24(r16) C L1
298:
299: bis r31, r31, r31 C L0 st slosh
300: mulq r19, r1, r11 C U1
301: bis r31, r31, r31 C L1 st slosh
302: addq r12, r21, r12 C U0 hi mul + carry
303:
304: cmpult r6, r13, r20 C L0 lo add => carry
305: bis r31, r31, r31 C U1 mt
306: lda r18, -1(r18) C L1 bookkeeping
307: subq r13, r12, r22 C U0 hi add => answer
308:
309: bis r31, r31, r31 C U1 mt
310: cmpult r13, r12, r21 C L0 hi add => carry
311: addq r14, r20, r14 C U0 hi mul + carry
312: ldq r6, 16(r16) C L1
313:
314: bis r31, r31, r31 C U1 mt
315: subq r7, r15, r23 C L0 lo + acc
316: addq r14, r21, r14 C U0 hi mul + carry
317: ldq r0, 32(r17) C L1
318:
319: umulh r19, r1, r12 C U1
320: cmpult r7, r15, r20 C L0 lo add => carry
321: subq r23, r14, r28 C U0 hi add => answer
322: ldq r7, 24(r16) C L1
323:
324: mulq r19, r2, r13 C U1
325: cmpult r23, r14, r21 C L0 hi add => carry
326: addq r8, r20, r8 C U0 hi mul + carry
327: ldq r1, 40(r17) C L1
328:
329: umulh r19, r2, r14 C U1
330: subq r4, r9, r9 C U0 lo + acc
331: stq r22, -16(r16) C L0
332: stq r28, -8(r16) C L1
333:
334: bis r31, r31, r31 C L0 st slosh
335: mulq r19, r3, r15 C U1
336: bis r31, r31, r31 C L1 st slosh
337: addq r8, r21, r8 C L0 hi mul + carry
338:
339: cmpult r4, r9, r20 C L0 lo add => carry
340: bis r31, r31, r31 C U1 mt
341: lda r17, 64(r17) C L1 bookkeeping
342: subq r9, r8, r22 C U0 hi add => answer
343:
344: bis r31, r31, r31 C U1 mt
345: cmpult r9, r8, r21 C L0 hi add => carry
346: addq r10, r20, r10 C U0 hi mul + carry
347: ldq r4, 32(r16) C L1
348:
349: bis r31, r31, r31 C U1 mt
350: subq r5, r11, r23 C L0 lo + acc
351: addq r10, r21, r10 C L0 hi mul + carry
352: ldq r2, -16(r17) C L1
353:
354: umulh r19, r3, r8 C U1
355: cmpult r5, r11, r20 C L0 lo add => carry
356: subq r23, r10, r28 C U0 hi add => answer
357: ldq r5, 40(r16) C L1
358:
359: mulq r19, r0, r9 C U1
360: cmpult r23, r10, r21 C L0 hi add => carry
361: addq r12, r20, r12 C U0 hi mul + carry
362: ldq r3, -8(r17) C L1
363:
364: umulh r19, r0, r10 C U1
365: subq r6, r13, r13 C L0 lo + acc
366: stq r22, 0(r16) C L0
367: stq r28, 8(r16) C L1
368:
369: bis r31, r31, r31 C L0 st slosh
370: mulq r19, r1, r11 C U1
371: bis r31, r31, r31 C L1 st slosh
372: addq r12, r21, r12 C U0 hi mul + carry
373:
374: cmpult r6, r13, r20 C L0 lo add => carry
375: bis r31, r31, r31 C U1 mt
376: lda r16, 64(r16) C L1 bookkeeping
377: subq r13, r12, r22 C U0 hi add => answer
378:
379: bis r31, r31, r31 C U1 mt
380: cmpult r13, r12, r21 C L0 hi add => carry
381: addq r14, r20, r14 C U0 hi mul + carry
382: ldq r6, -16(r16) C L1
383:
384: bis r31, r31, r31 C U1 mt
385: subq r7, r15, r23 C L0 lo + acc
386: addq r14, r21, r14 C U0 hi mul + carry
387: ldq r0, 0(r17) C L1
388:
389: umulh r19, r1, r12 C U1
390: cmpult r7, r15, r20 C L0 lo add => carry
391: subq r23, r14, r28 C U0 hi add => answer
392: ldq r7, -8(r16) C L1
393:
394: mulq r19, r2, r13 C U1
395: cmpult r23, r14, r21 C L0 hi add => carry
396: addq r8, r20, r8 C U0 hi mul + carry
397: ldq r1, 8(r17) C L1
398:
399: umulh r19, r2, r14 C U1
400: subq r4, r9, r9 C L0 lo + acc
401: stq r22, -48(r16) C L0
402: stq r28, -40(r16) C L1
403:
404: bis r31, r31, r31 C L0 st slosh
405: mulq r19, r3, r15 C U1
406: bis r31, r31, r31 C L1 st slosh
407: addq r8, r21, r8 C U0 hi mul + carry
408:
409: cmpult r4, r9, r20 C L0 lo add => carry
410: subq r9, r8, r22 C U0 hi add => answer
411: bis r31, r31, r31 C L1 mt
412: bgt r18, $Loop C U1 bookkeeping
413:
414: C ____ UNROLLED LOOP SOFTWARE PIPELINE FINISH ____
415: $Lend:
416: cmpult r9, r8, r21 C L0 hi add => carry
417: addq r10, r20, r10 C U0 hi mul + carry
418: ldq r4, 0(r16) C L1
419: subq r5, r11, r23 C L0 lo + acc
420: addq r10, r21, r10 C L0 hi mul + carry
421: umulh r19, r3, r8 C U1
422: cmpult r5, r11, r20 C L0 lo add => carry
423: subq r23, r10, r28 C U0 hi add => answer
424: ldq r5, 8(r16) C L1
425: mulq r19, r0, r9 C U1
426: cmpult r23, r10, r21 C L0 hi add => carry
427: addq r12, r20, r12 C U0 hi mul + carry
428: umulh r19, r0, r10 C U1
429: subq r6, r13, r13 C L0 lo + acc
430: stq r22, -32(r16) C L0
431: stq r28, -24(r16) C L1
432: mulq r19, r1, r11 C U1
433: addq r12, r21, r12 C U0 hi mul + carry
434: cmpult r6, r13, r20 C L0 lo add => carry
435: subq r13, r12, r22 C U0 hi add => answer
436: cmpult r13, r12, r21 C L0 hi add => carry
437: addq r14, r20, r14 C U0 hi mul + carry
438: subq r7, r15, r23 C L0 lo + acc
439: addq r14, r21, r14 C U0 hi mul + carry
440: umulh r19, r1, r12 C U1
441: cmpult r7, r15, r20 C L0 lo add => carry
442: subq r23, r14, r28 C U0 hi add => answer
443: cmpult r23, r14, r21 C L0 hi add => carry
444: addq r8, r20, r8 C U0 hi mul + carry
445: subq r4, r9, r9 C U0 lo + acc
446: stq r22, -16(r16) C L0
447: stq r28, -8(r16) C L1
448: addq r8, r21, r8 C L0 hi mul + carry
449: cmpult r4, r9, r20 C L0 lo add => carry
450: subq r9, r8, r22 C U0 hi add => answer
451: cmpult r9, r8, r21 C L0 hi add => carry
452: addq r10, r20, r10 C U0 hi mul + carry
453: subq r5, r11, r23 C L0 lo + acc
454: addq r10, r21, r10 C L0 hi mul + carry
455: cmpult r5, r11, r20 C L0 lo add => carry
456: subq r23, r10, r28 C U0 hi add => answer
457: cmpult r23, r10, r21 C L0 hi add => carry
458: addq r12, r20, r12 C U0 hi mul + carry
459: stq r22, 0(r16) C L0
460: stq r28, 8(r16) C L1
461: addq r12, r21, r0 C U0 hi mul + carry
462:
463: ldq $9, 8($30)
464: ldq $10, 16($30)
465: ldq $11, 24($30)
466: ldq $12, 32($30)
467: ldq $13, 40($30)
468: ldq $14, 48($30)
469: ldq $15, 56($30)
470: lda $30, 240($30)
471: ret r31, (r26), 1
472: EPILOGUE(mpn_submul_1)
473: ASM_END()