HIPPI Folks, This is a summary of responses to a query that I sent to find out how to implement "IP and ARP on HIPPI" and have a good chance of interoperating. Following the summary are the responses, in some cases edited to a degree. I thank the people who responded and granted permission to include their responses: Laurent Julliard, Jeff Koniges, John Renwick, Thomas Skibo, Bob Doolittle, Ken Johnson, Tom Mortensen, Don Tolmie, Richard Thomsen, Jim Hughes, and Jonathan Hahn. In addition to the included responses, I've had helpful additional comments from Jim Hughes and others that I've not included. Please let me know if you see a need for corrections or improvements. -- Lansing Sloan ljsloan@llnl.gov Contents: Summary Notes Response from Hewlett-Packard Responses (2) from Cray Research Response from NetStar Responses (2) from Silicon Graphics Responses (2) from Thinking Machines Responses (2) from Network Systems Corporation Responses (3) from Maximum Strategy Comments from Jim Hughes, Network Systems Comments from John Renwick, NetStar (reply to Maximum Strategy) Comments from Richard Thomsen, LANL Comments from Don Tolmie, LANL Comments from Lansing Sloan, LLNL Comments from Jonathan Hahn (NASA Ames) Query as sent out **** Summary These are the results of a survey. They concern features of IP implementations on HIPPI. Compatibility of non-IP features are generally ignored. Compatibility with switches is not discussed. Not all vendors responded. The products covered by the responses are the following. Hewlett-Packard: HiPPI interface for H-P S700 workstation Cray Research: CRI hippi implementations on Cray X-MP, Cray Y-MP, Cray C-90, Cray YMP-ELs and Cray 2. (Not CRS sparc-based superserver.) NetStar: GigaRouter Silicon Graphics Inc.: Challenge HIPPI product Thinking Machines: CM5 Network Systems Corporation: HIPPI-DX Maximum Strategy: File Server product In this table, "y" means "yes", "n" means "no", "o" means "optional," and "?" probably means I asked the question badly. In some cases, I had to interpret responses, and may not have always been consistent. For more details, see the individual responses. Vendor M N a e x t S S t C t N S T r H R a S G M a P I r C I C t Original Questions (abbreviated) Problem if first burst is short? n n n n n n n Problem if last burst is short? n n n n n n n Problem if first burst not short? n n n n n n n Problem if last burst not short? n n n n n n n Problem if D2 starts in first burst? n n n n n n n Problem if D2 starts in second burst? n n n n n n n Send nonzero D2 offset n n n n n n n Send and can receive LLC and SNAP y y y y y y y Support for ARP? y n n n n n n Support for tables, ARP not needed? y y y y y y y HIPPI-LE header ignored for IP? y y y y y y n Can fill in IEEE Destination address? y n n n y y y Are any other fields filled in? y n y y n y y D1 area with HIPPI-LE sent? y y y y y y y D1 with no fill accepted? y y y y y y y D1 with no fill sent? y y y y y y y Able to receive if logical HIPPI address? y y y y y y y Able to send with logical HIPPI address? y y y y y y y Able to receive multiple IPs in connection y y y y y y y Mix IP and other within connection (send)? n ? n y n n n OK to distribute information? y y y y y y y Volunteered information (may not be consistent from one reply to the next) Multiple IP addresses per HIPPI address? o (see note 8) Send source IEEE address? y n y y n y y Use "any" I-field if W is correct? y y y Able to send "raw LLC"? y n Self-address-resolution support? y n n n n n n Send multiple IP packets in a connection? n n y n n Mix IP and other in connection (receive)? y o y y n Require LLC/SNAP with IP? y y y y y D1 fill acceptable? y y y y y Allow odd number of 32-bit words (note 10)? n n y **** Notes 1. Because of the way I worded my questions, some people responded "true" and I thus could not fill in some table entries. My fault. 2. The IOSC HILDA HIPPI analyzer captures only the first burst of each packet that goes by. This feature is useless if the TCP/IP header doesn't show up in the first burst. (Jonathan Hahn) 3. LANL reported that nonzero D2 offsets have been seen (not from the vendors above, however). A couple of the responses indicated they would be unhappy to receive nonzero D2 offsets. 4. Congratulations to H-P for supporting ARP! 5. Maximum Strategy plans to support ARP in the future. 6. I have not spelled out which fields of the HIPPI-LE header are filled in, except for the Destination IEEE address. If I were to send out a survey knowing what I know now, I'd improve this. 7. CRI is able to support multiple kinds of traffic at the discretion of a system administrator. It is unclear whether multiple types of traffic may be sent within a connection, though. 8. Thinking Machines pointed out that they can allow multiple IP addresses to be supported on a single HIPPI adapter using a single HIPPI address (or multiple HIPPI addresses). Since I started this query to try to find out how we could have similar freedom, it's odd I forgot to ask about the possibility. 9. Silicon Graphics reports there is a gateway that does not accept an odd number of 32-bit words (and thus is HIPPI-FP compliant). The SGI response has additional information. 10. HIPPI-FP specifies that an even number of 32-bit words shall be sent. Of course, the D2 area size field allows saying exactly how many bytes are meaningful. Sending an odd number of 32-bit words violates HIPPI-FP. A couple of responses have indicated that some interoperability problems arise when an odd number of 32-bit words is sent. 11. In some cases the table reflects what vendors expect will be true with fully released products, though it may not have been true as of this survey. 12. H-P includes D1 fill for self-address-resolution. 13. Richard Thomsen's comments are exceptionally useful concerning some of the things that have been seen. 14. Several vendors stated they could support logical addresses or source-route addresses equally well. No one said they could not support source-route addresses. (I did not ask.) 15. Several vendors mentioned other kinds of HIPPI traffic that they can support but (mostly) I did not include that in the tables above. This is sometimes scattered among the responses to the survey questions. **** Response from Hewlett-Packard >From laurentj@gnlab030.grenoble.hp.com Tue Mar 15 05:43:54 1994 To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Cc: Maziar Tamadon , jpc@gnlab030.grenoble.hp.com, Francois Gaullier Subject: Re: Query about IP and ARP on HIPPI Date: Tue, 15 Mar 94 14:43:47 +0100 Message-Id: <28018.763739027@gnlab032.grenoble.hp.com> From: Laurent JULLIARD LAB GND 1267 Hello Lansing, Here the answers to your questions. Those answers concern the HiPPI interface for Hewlett Packard S700 workstation. This interface and its driver will be available next month on the market. You can include the answers below in your summary. I'm interested in having the results of your survey. > Regarding short bursts and related issues, I am unclear. > > * Does anyone have a problem if the first burst is short? > * Does anyone have a problem if the last burst is short? > * Does anyone have a problem if the first burst is not short? > * Does anyone have a problem if the last burst is not short? > * Does anyone have a problem if the D2 area starts in the first burst? > * Does anyone have a problem if the D2 area starts in the second burst? > > * Does anyone send a nonzero D2 offset for IP? > The answer is NO for each point. > Regarding the D2-area, I think > > * All send, and can receive, LLC and SNAP. YES. As well as raw "LLC" traffic because we do offer a Programmatic Link Level (at ISO 802.2 level) Access. > > Regarding the D1-area and HIPPI-LE header for IP, I think > > * ARP is not used or supported by many, and not required. We fully support RFC 1374. It means that our HiPPI interface behaves exactly like any other network interface such as FDDI or ethernet. The ARP agent functionality is supported as well. > * Most implementations use tables in place of ARP. You can also use a hard coded table instead of (or in addition to) the dynamic ARP agent functionality. > * For IP, receivers ignore the HIPPI-LE header. Yes for IP. (but not for ARP request/reply). > * For IP, senders can fill in the destination IEEE address. Yes. > * For IP, senders fill in no other fields of the HIPPI-LE header. No. Our LE header is totally filled in. (MAC address, Switch address and type) > * For IP, a D1 area with HIPPI-LE header is sent in each packet. Yes. > * A D1-area with no fill is accepted (3 64-bit words) Yes. > * A D1-area with no fill is generated by at least some, Yes we generate D1 area with no fill. (except for AR_S_request and AR_S_response). I forgot to mention that we do support the self address resolution schema of RFC 1374. > > Regarding HIPPI addressing in the I-field, I think > > * All implementations can receive when logical HIPPI addresses are used. > * All implementations can send using logical HIPPI addresses. Yes. > > Regarding HIPPI connections, I think > > * Within reason, multiple IP packets can be received on a connection. Yes we can receive multiple packets in a connection. But as recommended in RFC 1374 we never send several packets in a connection. > * Within a connection, distinct kinds of traffic are not sent. > (E.g., all IP, all IPI, or all whatever would be sent.) No we never send multiple packets in a single connection but we can receive such kind of traffic even though the packets are not of the same nature). Feel free to contact me if your want further information. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ Laurent JULLIARD (Box 26) | HPDesk: Laurent Julliard /HP6300/UM ~ ~ GND/High Speed Networking Lab | Unix: Laurent_Julliard@grenoble.hp.com~ ~ HEWLETT-PACKARD FRANCE | Phone: (33) 76 62 12 67 ~ ~ 5, avenue Raymond Chanas - EYBENS | Telnet: 779 12 67 ~ ~ 38053 GRENOBLE CEDEX 9 | Fax: (33) 76 62 12 86 ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **** Responses (2) from Cray Research Second response from Cray Research >From jeffk@sdiv.cray.com Fri Apr 1 05:44:31 1994 From: jeffk@sdiv.cray.com (Jeff Koniges) Message-Id: <9404011344.AA17736@media08> Subject: Re: hippi survey To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Date: Fri, 1 Apr 1994 07:44:15 -0600 (CST) Lance, Well, it figures. Right after I told you to delete the CRI information, I got approval from my manager to include it. Sorry for the delay. -- Jeff First response from Cray Research >From jeffk@palm.cray.com Tue Mar 15 07:31:30 1994 From: jeffk@palm.cray.com (Jeff Koniges) Message-Id: <9403151531.AA10410@media08.cray.com> Subject: Re: Query about IP and ARP on HIPPI To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Date: Tue, 15 Mar 94 9:31:02 CST Lansing, My replies cover the CRI hippi implementations on Cray X-MP, Cray Y-MP, Cray C-90, Cray YMP-ELs and Cray 2. I don't have any information on the CRS (Sparc-based superserver) stuff. > > > > Please reply directly to my e-mail address, not to the mail > group. If you want to know the results, tell me: I'll be willing to > send out a summary. (If you respond but don't want your > information included, please state this.) I'd be interested in receiving a summary. > > Regarding short bursts and related issues, I am unclear. > > * Does anyone have a problem if the first burst is short? > * Does anyone have a problem if the last burst is short? > * Does anyone have a problem if the first burst is not short? > * Does anyone have a problem if the last burst is not short? As long as HIPPI-PH is not violated, i.e. no packet with two short bursts, we can receive packets with the short burst at the start or at the end. We always send short bursts at the end. > * Does anyone have a problem if the D2 area starts in the first burst? > * Does anyone have a problem if the D2 area starts in the second burst? We can accept packets with the D2 area in either place as long as the D1 size in the FP header can be used to locate the start of the D2 area. > > * Does anyone send a nonzero D2 offset for IP? We don't. > > Regarding the D2-area, I think > > * All send, and can receive, LLC and SNAP. For IP, we send both and require both. > > Regarding the D1-area and HIPPI-LE header for IP, I think > > * ARP is not used or supported by many, and not required. > * Most implementations use tables in place of ARP. We don't support ARP over HIPPI. We use tables. > * For IP, receivers ignore the HIPPI-LE header. True, we ignore it. > * For IP, senders can fill in the destination IEEE address. We don't fill in any fields in the HIPPI-LE header. > * For IP, senders fill in no other fields of the HIPPI-LE header. We don't fill in any fields in the HIPPI-LE header. > * For IP, a D1 area with HIPPI-LE header is sent in each packet. We send the header, but it's all zeros. > * A D1-area with no fill is accepted (3 64-bit words) > * A D1-area with no fill is generated by at least some, We send a 3 word (64 bit) D1-area and accept any size. > > Regarding HIPPI addressing in the I-field, I think > > * All implementations can receive when logical HIPPI addresses are used. > * All implementations can send using logical HIPPI addresses. We can receive an incoming packet with any i-field, except that the double-wide bit must be correct: Set only if double-wide mode is being used. The same is true for sending: we will use whatever I-Field is configured for a given host, but the double-wide bit must be correct. Double-wide is only supported on CRI systems with an IOS Model E. > > Regarding HIPPI connections, I think > > * Within reason, multiple IP packets can be received on a connection. We can receive an unlimited number. > * Within a connection, distinct kinds of traffic are not sent. > (E.g., all IP, all IPI, or all whatever would be sent.) We do not support the mixing of IP and IPI traffic on the same HIPPI channel pair. We allow sharing of the HIPPI channel between IP traffic and user applications, at the discretion of the system administrator, with the restriction that sharing applications conform to HIPPI-FP. This sharing is not connection related. Whether packets come in on the same connection or on different connections makes no difference. > > Thanks. > > Lansing Sloan, LLNL > ljsloan@llnl.gov > -- Jeff Koniges Cray Research, Inc. jeffk@cray.com **** Responses (2) from NetStar Second NetStar response (edited) >From jkr@netstar.com Tue Mar 22 14:59:05 1994 From: jkr@netstar.com (John Renwick) Message-Id: <9403222259.AA28329@banzai.netstar.com> Subject: Re: hippi-ip survey (second draft) To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Date: Tue, 22 Mar 1994 16:59:13 -0600 (CST) Lance, I'd like to alter my response to your query. We'll be supporting short-burst-first when the GigaRouter is fully released this summer, so please change the "y"s to "n" here and change my responses to a simple "No" for these questions. With that change, you may distribute my responses. I've added some "y"s and an "n" in the stuff below: > > OK to distribute information? y ? y y ? y ? <---- > Want to see information? y y y y ? y ? > > Volunteered information (may not be > consistent from one reply to the next) > > Multiple IP addresses per HIPPI address? o > (see note 8) > Send source IEEE address? y n y y n y y > Use "any" I-field if W is correct? y y y <---- > Able to send "raw LLC"? y n > Self-address-resolution support? y n n n n n n > Send multiple IP packets in a connection? n n y n n > Mix IP and other in connection (receive)? y o y n y n > Require LLC/SNAP with IP? y y y y <---- > D1 fill acceptable? y y y <---- > Allow odd number of 32-bit words (note 10)? n n y <---- > -- John K. Renwick NetStar, Inc. ph. (612) 943-8990 jkr@netstar.com 10250 Valley View Rd FAX (612) 943-8939 Eden Prairie, MN 55344 First NetStar response (edited) >From jkr@netstar.com Tue Mar 15 07:32:21 1994 From: jkr@netstar.com (John Renwick) Message-Id: <9403151532.AA27400@banzai.netstar.com> Subject: Re: Query about IP and ARP on HIPPI To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Date: Tue, 15 Mar 1994 09:32:24 -0600 (CST) Lance, I'd like to know anything you find out from this query. I'll answer for the NetStar GigaRouter (effective when the GigaRouter is fully released this summer): > * Does anyone have a problem if the first burst is short? No. > * Does anyone have a problem if the last burst is short? No. > * Does anyone have a problem if the first burst is not short? No. > * Does anyone have a problem if the last burst is not short? No. > * Does anyone have a problem if the D2 area starts in the first burst? No. > * Does anyone have a problem if the D2 area starts in the second burst? No. > > * Does anyone send a nonzero D2 offset for IP? No. The RFC discourages this. It's generally detrimental to performance. > > Regarding the D2-area, I think > > * All send, and can receive, LLC and SNAP. Yes. The RFC requires it. > > Regarding the D1-area and HIPPI-LE header for IP, I think > > * ARP is not used or supported by many, and not required. I don't know of any ARP implementations. We don't have it. > * Most implementations use tables in place of ARP. All do, as far as I know. > * For IP, receivers ignore the HIPPI-LE header. I do. > * For IP, senders can fill in the destination IEEE address. If they know it. Without ARP, it isn't defined. I fill in the source IEEE address (yes, our HIPPI boards have IEEE addresses). > * For IP, senders fill in no other fields of the HIPPI-LE header. I don't use any. > * For IP, a D1 area with HIPPI-LE header is sent in each packet. Yes. The RFC requires this. > * A D1-area with no fill is accepted (3 64-bit words) Yes. The RFC says you "should" send it this way. > * A D1-area with no fill is generated by at least some, All, as far as I know. > > Regarding HIPPI addressing in the I-field, I think > > * All implementations can receive when logical HIPPI addresses are used. Yes. The destination really has no reason to care about the I-field. > * All implementations can send using logical HIPPI addresses. Ours can. > > Regarding HIPPI connections, I think > > * Within reason, multiple IP packets can be received on a connection. Our destination doesn't care how many packets arrive on one connection. I don't send more than one IP packet per connection. The RFC limits the connection to 68 bursts unless source and destination agree on something else. > * Within a connection, distinct kinds of traffic are not sent. > (E.g., all IP, all IPI, or all whatever would be sent.) Yes. > > Thanks. My pleasure! If you hear from anyone that's implementing ARP, please let me know. Also, RFC 1374 says you "should" construct packets with no D1 fill, no D2 offset, and the short burst (if any) last, with no more than 7 bytes of fill at the end. D2-on-burst- boundary must never be set. (See the RFC, page 11.) If you hear of anyone doing anything different, could you please let me know? Such implementations should be discouraged. - John -- John K. Renwick NetStar, Inc. ph. (612) 943-8990 jkr@netstar.com 10250 Valley View Rd FAX (612) 943-8939 Eden Prairie, MN 55344 **** Responses (2) from Silicon Graphics Second response from Silicon Graphics From skibo@florida.wpd.sgi.com Thu Mar 24 17:57:03 1994 Date: Thu, 24 Mar 94 17:56:59 -0800 From: skibo@florida.wpd.sgi.com (Thomas Skibo) Message-Id: <9403250156.AA08861@florida.wpd.sgi.com> To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Subject: Re: hippi survey > From ljsloan@ocfmail.ocf.llnl.gov Thu Mar 24 12:56:45 1994 > > Tom, > > Is it acceptable to distribute the SGI information you've provided? Yes. No problem. --- Thomas Skibo Media Systems Division skibo@sgi.com Silicon Graphics, Inc. First response from Silicon Graphics >From skibo@florida.wpd.sgi.com Tue Mar 15 11:04:30 1994 Date: Tue, 15 Mar 94 11:04:14 -0800 From: skibo@florida.wpd.sgi.com (Thomas Skibo) Message-Id: <9403151904.AA04414@florida.wpd.sgi.com> To: ljsloan@llnl.gov Subject: Re: Hippi query Cc: skibo@florida.wpd.sgi.com, greg@florida.wpd.sgi.com Hello. I was forwarded your message and thought I'd give a quick reply and let you know a little about our Challenge HIPPI product. > >From hippi-admin@Think.COM Tue Mar 15 00:50:32 1994 > From: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) > To: hippi-ext@Think.COM, ljsloan@ocfmail.ocf.llnl.gov > Subject: Query about IP and ARP on HIPPI > Cc: det@lanl.gov > Sender: hippi-admin@Think.COM > > > HIPPI folks, > > I'm looking into what is required for interoperability among > HIPPI implementations, particularly for IP. LLNL is acquiring > a HIPPI adapter that probably cannot fully support the HIPPI standard, > and would like to know what the tradeoffs are to be as interoperable > as possible. > > I've listed some questions below and what I think the answers > may be. Anyone please let me know if I'm right and, if not, > let me know what is correct. > > Please reply directly to my e-mail address, not to the mail > group. If you want to know the results, tell me: I'll be willing to > send out a summary. (If you respond but don't want your > information included, please state this.) > > Regarding short bursts and related issues, I am unclear. > > * Does anyone have a problem if the first burst is short? > * Does anyone have a problem if the last burst is short? > * Does anyone have a problem if the first burst is not short? > * Does anyone have a problem if the last burst is not short? > * Does anyone have a problem if the D2 area starts in the first burst? > * Does anyone have a problem if the D2 area starts in the second burst? No problems here. > * Does anyone send a nonzero D2 offset for IP? We don't. And, we would choke if we received such a packet. > Regarding the D2-area, I think > > * All send, and can receive, LLC and SNAP. Yes. A note about D2-area: I know there is a gateway out there that can only receive D2-area's that are 64-bit multiples. It turns out they are correct: the FP standard says D2-area should always be a double-word multiple. For a while, some machines, including our early prototypes, would send IP packets with odd-word length D2-area's. I think this has been corrected by everybody. Not only that, but the manufacturer of the box might make it more "liberal in what it accepts" by handling the odd-word case. > Regarding the D1-area and HIPPI-LE header for IP, I think > > * ARP is not used or supported by many, and not required. > * Most implementations use tables in place of ARP. > * For IP, receivers ignore the HIPPI-LE header. > * For IP, senders can fill in the destination IEEE address. > * For IP, senders fill in no other fields of the HIPPI-LE header. > * For IP, a D1 area with HIPPI-LE header is sent in each packet. > * A D1-area with no fill is accepted (3 64-bit words) > * A D1-area with no fill is generated by at least some, Correct. > > Regarding HIPPI addressing in the I-field, I think > > * All implementations can receive when logical HIPPI addresses are used. Correct. We pretty much ignore I-fields that are received. > * All implementations can send using logical HIPPI addresses. Correct. We don't interpret I-fields put in our IP/I-field table so you just put in logical address I-fields just as easily as source route I-fields. > Regarding HIPPI connections, I think > > * Within reason, multiple IP packets can be received on a connection. > * Within a connection, distinct kinds of traffic are not sent. > (E.g., all IP, all IPI, or all whatever would be sent.) Correct. > Thanks. > > Lansing Sloan, LLNL > ljsloan@llnl.gov Hope it helps! --- Thomas Skibo Media Servers Division / Advanced Networking skibo@sgi.com Silicon Graphics, Inc. **** Responses (2) from Thinking Machines Second TMC response >From rad@Think.COM Tue Mar 22 12:16:59 1994 Message-Id: <9403222016.AA05744@avenger.think.com> To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Subject: Re: draft summary of hippi-ip responses Date: Tue, 22 Mar 94 15:16:52 EST From: Bob Doolittle Lansing, you got my responses down correctly, except for the following: Mix IP and other within connection (send)? n ? n ? n ? n We don't do this. OK to distribute information? y ? ? ? ? ? ? Fine. Want to see information? y y y y ? ? ? Absolutely. Addresses map one-to-one? o We don't do this, for either ifield or IEEE addresses. They can be one-to-one, many-to-one, or one-to-many. Use "any" I-field if W is correct? y Yes, for IP. Large FP packets may be aborted if they are sent to us with an ifield we are not configured to receive. The packet is accepted, and then once it is delivered to the ULP we check the ifield, and if it isn't configured, or there is no ULP or application which has "bound" to the ifield, we tell the HIPPI to drop Connect (so as to not be flooded with data in a huge (or infinite length) uninteresting packet). An IP packet will always be delivered atomically, so this isn't an issue for IP (it's too late to drop Connect before we determine that the ifield isn't acceptable in a 64 Kbyte packet). Able to send "raw LLC"? y No. Only IP, raw HIPPI-FP, and raw HIPPI-PH. We can receive any mix of these types simultaneously (HIPPI-PH must come with an ifield for which we are configured to *always* expect HIPPI-PH). Self-address-resolution support? y No. Send multiple IP packets in a connection? n n n No. Mix IP and other in connection (receive)? y o n n n Fine, for either raw FP or IP. Require LLC/SNAP with IP? y Yes, although we don't use the information, other than ethertype. The RFC requires something be there. D1 fill acceptable? Yes. Allow odd number of 32-bit words? Yes, although we never generate an odd number of 32-bit words in an FP packet (this violates HIPPI-FP). -Bob First TMC response >From rad@Think.COM Tue Mar 15 18:19:47 1994 Message-Id: <9403160219.AA02517@avenger.think.com> To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Cc: hippi-imps@Think.COM Subject: Re: Query about IP and ARP on HIPPI Date: Tue, 15 Mar 94 21:19:37 EST From: Bob Doolittle Lansing, we fall into your expected categories in (almost) all cases, with the following comments: * Does anyone have a problem if the first burst is short? The CM5-HIPPI never sends this, but we have no problem receiving it. * Does anyone send a nonzero D2 offset for IP? We don't. Please tell us if anyone responds yes to this! We will be unhappy to hear it :-). * For IP, senders fill in no other fields of the HIPPI-LE header. We fill in our source IEEE address from our arp table. * For IP, a D1 area with HIPPI-LE header is sent in each packet. This is required by RFC1374, I think. * A D1-area with no fill is generated by at least some, We do this. I wonder if anybody fills D1 for IP? We also have one unexpected characteristic you didn't point out, but which deserves mentioning - we do not necessarily have a one-to-one mapping of ifield to IP address, although we can be configured in that way using multiple logical addresses for our adapter. In other words, a single HIPPI adapter may be shared amongst multiple IP hosts within the CM5. We peek at the IP header to dispatch it within the CM5. If users wish, they may assign multiple logical addresses to our adapter for our multiple IP hosts, but there is no requirement to do so. We may or may not use multiple IEEE addresses for a single HIPPI adapter as well, for the same reasons. It depends on how the ARP table is configured. -Bob ----------------------------------------------------------------------------- -- Bob Doolittle Thinking Machines Corporation (617) 234-2734 245 First Street rad@think.com Cambridge, MA 02142 ----------------------------------------------------------------------------- -- **** Responses (2) from Network Systems Second NSC response >From johnskr@anubis.network.com Tue Mar 22 06:02:16 1994 From: johnskr@anubis.network.com (Ken Johnson) Message-Id: <9403221402.AA06591@anubis.network.com> Subject: Re: draft summary of hippi-ip responses To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Date: Tue, 22 Mar 1994 08:02:03 -0600 (CST) Cc: hughes@anubis.network.com (Jim Hughes), johnskr@anubis.network.com (Ken Johnson) Lansing, Let me remove some of the ? marks in your survey from NSC. Support for ARP? no. Support for tables, ARP not needed? yes. If what you mean is that the tables provide the I-field of a particular IP address. Mix IP and other within connection (send)? yes. OK to distribute information? yes. Let me correct some responses. Send multiple IP packets in a connection? yes. Mix IP and other in connection (receive)? yes, but only IP and HYPERchannel data are currently supported. Require LLC/SNAP with IP? yes. D1 fill acceptable? yes. Allow odd number of 32-bit words? Not in physical packet, but FP- header can describe any byte length of D2. Ken Johnson First NSC response >From johnskr@anubis.network.com Wed Mar 16 09:48:12 1994 From: johnskr@anubis.network.com (Ken Johnson) Message-Id: <9403161746.AA12642@anubis.network.com> Subject: Re: HIPPI questions To: ljsloan@ocfmail.ocf.llnl.gov Date: Wed, 16 Mar 1994 11:46:14 -0600 (CST) Cc: hughes@hughes.network.com, johnskr@anubis.network.com (Ken Johnson) Mr. Sloan, My name is Ken Johnson. I work for Network Systems and did the HIPPI-DX firmware. The answers are based upon the current released code. Regarding short bursts and related issues, I am unclear. * Does anyone have a problem if the first burst is short? No. Data flows into a fifo and burst markers are lost. * Does anyone have a problem if the last burst is short? No. * Does anyone have a problem if the first burst is not short? No. * Does anyone have a problem if the last burst is not short? No. * Does anyone have a problem if the D2 area starts in the first burst? No. * Does anyone have a problem if the D2 area starts in the second burst? No. * Does anyone send a nonzero D2 offset for IP? No. Regarding the D2-area, I think * All send, and can receive, LLC and SNAP. Yes. In fact, the packet must contain a SNAP header. Regarding the D1-area and HIPPI-LE header for IP, I think * ARP is not used or supported by many, and not required. True. * Most implementations use tables in place of ARP. True. * For IP, receivers ignore the HIPPI-LE header. True. * For IP, senders can fill in the destination IEEE address. False. Only fill the source address is filled. * For IP, senders fill in no other fields of the HIPPI-LE header. False. Only fill the source address is filled. * For IP, a D1 area with HIPPI-LE header is sent in each packet. True. Required. * A D1-area with no fill is accepted (3 64-bit words) True. * A D1-area with no fill is generated by at least some, True. No-fill ever sent. Regarding HIPPI addressing in the I-field, I think * All implementations can receive when logical HIPPI addresses are used. True. Received I-fields are ignored. In fact, due to speed, some I-fields are lost and not seen. To be able to read and save all of them would require the firmware to "manually" detect each Request and "manually" send each Connect. This would reduce our packets per second by 1/2. * All implementations can send using logical HIPPI addresses. True. Table driven. Regarding HIPPI connections, I think * Within reason, multiple IP packets can be received on a connection. True. The connection need never drop. * Within a connection, distinct kinds of traffic are not sent. (E.g., all IP, all IPI, or all whatever would be sent.) True. A packet is our logical unit. Each packet is treated separately. Ken Johnson **** Responses (3) from Maximum Strategy Third response from Maximum Strategy >From tomm@maxstrat.com Wed Mar 30 13:03:09 1994 Date: Wed, 30 Mar 94 12:42:28 PST From: Tom Subject: RE: error in HIPPI-IP survey response? To: Lansing J Sloan Message-Id: Lansing, [material deleted] After re-reading the RFC, I noticed the phrase on page 6 which reads: (referring to D1_Area_Size) "Destinations shall accept any value that HIPPI-FP defines as legal..." We went ahead and made the changes necessary to support fill in D1. However, our hardware is designed to optimize header processing, but only for data received in the first burst of a HIPPI packet. So, while we do support packets with short first bursts and/or fill in the first burst, the highest performance is realized when all the headers reside in the first burst. [material deleted] -Tom Mortensen (tomm@maxstrat.com) 03/30/94 12:42:28 Second response from Maximum Strategy From tomm@maxstrat.com Thu Mar 24 15:55:05 1994 Date: Thu, 24 Mar 94 15:44:52 PST From: Tom Subject: RE: hippi survey To: Lansing J Sloan Message-Id: To: Lansing Sloan, and John Renwick. >Is it acceptable to distribute the Max Strat information you've >provided? Yes. [material deleted, see "Comments from John Renwick, NetStar".] Tom Mortensen Maximum Strategy, Inc. First response from Maximum Strategy >From tomm@maxstrat.com Wed Mar 16 12:56:18 1994 Date: Wed, 16 Mar 94 12:26:58 PST From: Tom Subject: RE: Query about IP and ARP on HIPPI To: Lansing J Sloan Message-Id: Dear Mr. Sloan, Here is our response to your questions concerning IP on HIPPI. The answers to these questions are for the Maximum Strategy File Server product. This machine provides NFS for the high performance network environment. It is able to process up to 800 60K I/O requests per second using NFS-3 via IP (either UDP/IP or TCP/IP). If you desire additional information regarding this product, let me know. Also, we would be very interested in obtaining the summary of answers which you will be generating. Now, on with our answers... >Regarding short bursts and related issues, I am unclear. > >* Does anyone have a problem if the first burst is short? >* Does anyone have a problem if the last burst is short? >* Does anyone have a problem if the first burst is not short? >* Does anyone have a problem if the last burst is not short? We will support shorts bursts either at the beginning or the end of a packet (that is, either the 1st burst or the last burst can be a short burst, but not both). >* Does anyone have a problem if the D2 area starts in the first burst? >* Does anyone have a problem if the D2 area starts in the second burst? The only way D2 can start in the 2nd burst would be if the 1st burst is a short burst of exactly 32 bytes. In this case, we would process the packet correctly. >* Does anyone send a nonzero D2 offset for IP? We never do. >Regarding the D2-area, I think >* All send, and can receive, LLC and SNAP. Yes, LLC/SNAP is expected and must be present (or else error). >Regarding the D1-area and HIPPI-LE header for IP, I think >* ARP is not used or supported by many, and not required. Correct. >* Most implementations use tables in place of ARP. Correct again, although we do have plans to eventually implement ARP. >* For IP, receivers ignore the HIPPI-LE header. No. We check some fields: W M_Type D_A_T Dest. Switch Address (if D_A_T specifies logical) >* For IP, senders can fill in the destination IEEE address. Correct. >* For IP, senders fill in no other fields of the HIPPI-LE header. No, when sending a packet, we fill in these fields: W M_Type Dest. Switch Address Source Switch Address D_A_T S_A_T >* For IP, a D1 area with HIPPI-LE header is sent in each packet. Correct. This is a "must". >* A D1-area with no fill is accepted (3 64-bit words) This is the only size D1 we accept. >* A D1-area with no fill is generated by at least some, Correct. >Regarding HIPPI addressing in the I-field, I think >* All implementations can receive when logical HIPPI addresses are used. >* All implementations can send using logical HIPPI addresses. Yes to both. >Regarding HIPPI connections, I think >* Within reason, multiple IP packets can be received on a connection. Currently, we will accept any number of packets sent to us on a single connection. However, when we send packets out, we only send one packet per connection. >* Within a connection, distinct kinds of traffic are not sent. > (E.g., all IP, all IPI, or all whatever would be sent.) Correct. We currently only accept and generate IP for this product. >Thanks. Your welcome. -Tom (tomm@maxstrat.com) 03/16/94 12:26:58 **** Comments from Jim Hughes, Network Systems >From hughes@hughes.network.com Tue Mar 15 08:24:18 1994 From: hughes@hughes.network.com (James P. Hughes) Message-Id: <9403151032.ZM744@hughes.network.com> Date: Tue, 15 Mar 1994 10:32:07 -0600 To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) Subject: Re: Query about IP and ARP on HIPPI On Mar 15, 12:20am, Lansing J Sloan wrote: > Subject: Query about IP and ARP on HIPPI > > HIPPI folks, > > I'm looking into what is required for interoperability among > HIPPI implementations, particularly for IP. LLNL is acquiring > a HIPPI adapter that probably cannot fully support the HIPPI standard, > and would like to know what the tradeoffs are to be as interoperable > as possible. > > I've listed some questions below and what I think the answers > may be. Anyone please let me know if I'm right and, if not, > let me know what is correct. > > Please reply directly to my e-mail address, not to the mail > group. If you want to know the results, tell me: I'll be willing to > send out a summary. (If you respond but don't want your > information included, please state this.) Please send me a summary. > Regarding short bursts and related issues, I am unclear. > > * Does anyone have a problem if the first burst is short? > * Does anyone have a problem if the last burst is short? > * Does anyone have a problem if the first burst is not short? > * Does anyone have a problem if the last burst is not short? > * Does anyone have a problem if the D2 area starts in the first burst? > * Does anyone have a problem if the D2 area starts in the second burst? Although I do not have any first hand experience, to my knowledge, all of the existing implementations have a - non-short first burst and a - short last burst (unless the data falls on a 1k boundary) and the - D2 area is in the first burst. I expect that there may be some interoperability problems if you vary from the norm, but I would guess that no one will say so. > * Does anyone send a nonzero D2 offset for IP? Not to my knowledge. If the stuff comes in not byte aligned, then there will be at least performance problems. > Regarding the D2-area, I think > > * All send, and can receive, LLC and SNAP. Yes, and most with null IEEE 48 bit addresses... > Regarding the D1-area and HIPPI-LE header for IP, I think > > * ARP is not used or supported by many, and not required. I do not know of any implementations. > * Most implementations use tables in place of ARP. Yes. > * For IP, receivers ignore the HIPPI-LE header. Yes. > * For IP, senders can fill in the destination IEEE address. Some do. HP does (or asks you for that information in their ARP table). > * For IP, senders fill in no other fields of the HIPPI-LE header. Maybe LANL messes with the security field, but I doubt it. > * For IP, a D1 area with HIPPI-LE header is sent in each packet. yes > * A D1-area with no fill is accepted (3 64-bit words) yes > * A D1-area with no fill is generated by at least some, I would guess all.... > > Regarding HIPPI addressing in the I-field, I think > > * All implementations can receive when logical HIPPI addresses are used. to my knowledge. > * All implementations can send using logical HIPPI addresses. to my knowledge. > > Regarding HIPPI connections, I think > > * Within reason, multiple IP packets can be received on a connection. to my knowledge. > * Within a connection, distinct kinds of traffic are not sent. > (E.g., all IP, all IPI, or all whatever would be sent.) On the Cray, distinct types of traffic are not shared on the channel. That is, if a channel is an IP channel, then it can not be used for IPI3. This is a software restriction. The 2 channels can share a switch, but not the connections. I will also send this to out HIPPI-DX person to get his definitive answers. jim Comments from John Renwick, NetStar (reply to Maximum Strategy) >From jkr@netstar.com Fri Mar 25 06:44:02 1994 From: jkr@netstar.com (John Renwick) Message-Id: <9403251444.AA05980@banzai.netstar.com> Subject: Re: Comments on RFC 1374 To: tomm@maxstrat.com (Tom) Date: Fri, 25 Mar 1994 08:44:04 -0600 (CST) Cc: ljsloan@llnl.gov Tom, I'm glad to answer your HIPPI questions, but I can't help with IP or NFS over TCP/IP. For an authoritative answer to those questions, you might ask Dave Borman, dab@cray.com. > > Regarding IP over HIPPI > > 1. Does anyone send HIPPI packets with D2_Size of 0xFFFFFFFF ? > [We require D2_Size to be exact.] So does the RFC, at the bottom of page 6/top of page 7. > 2. Does anyone use the "source routing" I-Field format? > [Since we use a table to translate destination IP address to > I-Field, this does not matter to us.] Most likely. It all depends on how the site decides to operate the HIPPI switch. If the switch isn't programmable, like an NSC PS-8, you have to use source routes. I think that hosts usually don't care what the I-field contents are. > 3. Is there always a single IP packet within a HIPPI packet, or can > there be multiple IP packets within a HIPPI packet? No, HIPPI-LE doesn't support more than one IP packet in a HIPPI packet. > 4. Does anyone send more than 65 bursts in a single HIPPI packet? > 4b. If so, are these bursts simply "fill"? Not that I know of. The maximum size of an IP datagram on HIPPI is 65280, according to the RFC. This requires 64 bursts, or 65 if you send a short first burst containing only D1. If you sent a 66th burst, it would have to be all fill, which would seem sort of silly. It might also overflow the receiver's buffer, causing an error and loss of the packet. > 5. Does anyone send multiple HIPPI packets in a single connection? > 5b. If so, is the "maximum of 68 bursts per packet" rule strictly > followed? Most people seem to be sending just one IP datagram per connection. Some implementations, such as Cray Research's, can keep the connection up for an unlimited number of datagrams if the HIPPI link is configured as point-to-point. I don't know of anyone who's trying to send more than one datagram per connection but limiting the connection to some number of bursts. The RFC's 68-burst rule isn't absolute--you can ignore it if all hosts agree to something different. -- John K. Renwick NetStar, Inc. ph. (612) 943-8990 jkr@netstar.com 10250 Valley View Rd FAX (612) 943-8939 Eden Prairie, MN 55344 **** Comments (abridged) from Richard Thomsen, LANL From hippi-admin@think.com Thu Mar 24 10:08:02 1994 Date: Thu, 24 Mar 1994 07:56:40 -0700 From: rgt@lanl.gov (Richard Thomsen) Message-Id: <9403241456.AA08526@spitfire.lanl.gov> To: hippi-ext@think.com, hippi-ietf@cray.com Subject: RE: Comments on RFC 1374, IP and ARP on HIPPI Working on our HIPPI network at Los Alamos, we have seen machines that send short first bursts, as well as full first bursts of just FP and D1 with padding to the end of the burst, and D2 starting at the second burst. D1 Area Size may or may not indicate all this padding (the B bit is set). We have also seen padding at the end of D2, outside of D2 size. We have also seen a machine that used the D2 offset to pad out their data to word boundaries. That caused us much grief, but it was only one special machine. Our CBI always sends the short burst last, but has passed padding on after the D2 area. This has been fixed. Richard Thomsen Los Alamos National Laboratory Network Engineering (CIC-5) rgt@lanl.gov **** Comments (abridged) from Don Tolmie, LANL >From det@lanl.gov Mon Mar 14 09:57:56 1994 Message-Id: <199403141756.KAA25404@mailhost.lanl.gov> Date: Mon, 14 Mar 1994 10:57:50 -0700 To: ljsloan@ocfmail.ocf.llnl.gov (Lansing J Sloan) From: det@lanl.gov (Don Tolmie) Subject: Re: draft query to HIPPI folks Lansing, [Suggested improvement for the query deleted.] We agree with most of your conjectures, but have issue with one. >Regarding the D2-area, I think > >* No one sends a nonzero D2 offset for IP. (Richard Thomsen said that he has seen [a machine] use a nonzero offset.) Please let me know, or publish on the reflector, the results of your query. Don [I deleted the rest of Don's reply. It was a reply to an early form of my query.] **** Comments from Lansing Sloan, LLNL Unfortunately, we cannot contribute much in the way of interoperability experience for IP here. In one HIPPI subnetwork, we've had Cray Y-MPs IP with one another. In another HIPPI subnetwork, we've had IBM RS-6000's IP with one another. (This is the National Storage Laboratory HIPPI subnet, which uses IPI mostly.) Both subnets use Network Systems Corporation HIPPI switches. One uses source routing, the other uses logical addresses. That's about it for interoperability attempts so far. **** Comments from Jonathan Hahn (NASA Ames) >From hahn@nas.nasa.gov Mon Mar 21 18:52:32 1994 Date: Mon, 21 Mar 94 18:52:29 -0800 From: hahn@nas.nasa.gov (Jonathan Hahn) Message-Id: <9403220252.AA19702@gigantor.nas.nasa.gov> To: ljsloan@ocfmail.ocf.llnl.gov Subject: Re: Query about IP and ARP on HIPPI > * Does anyone have a problem if the D2 area starts in the first burst? > * Does anyone have a problem if the D2 area starts in the second burst? RFC 1374 mandates that the Start_D2_on_Burst_Boundary bit is zero. I.e. D2 must start in the first burst. The IOSC HILDA HIPPI analyzer captures only the first burst of each packet that goes by... this feature is useless if the TCP/IP header doesn't show up in the first burst. > * A D1-area with no fill is accepted (3 64-bit words) > * A D1-area with no fill is generated by at least some, One thing you didn't mention is D2-area fill. We had some boards in from Chi Systems that didn't generate D2-area fill under some circumstances. These boards could talk to all our machines (Convex, Cray C-90) except our Intel Paragon which didn't consider that the packet arrived until all the fill bytes arrived too. Chi has since fixed the problem in their S-bus board; haven't been able to check their VME board yet. Please send me a copy of your results if you don't post them to the list. -jonathan hahn NASA Ames Research Center **** Query as sent out HIPPI folks, I'm looking into what is required for interoperability among HIPPI implementations, particularly for IP. LLNL is acquiring a HIPPI adapter that probably cannot fully support the HIPPI standard, and would like to know what the tradeoffs are to be as interoperable as possible. I've listed some questions below and what I think the answers may be. Anyone please let me know if I'm right and, if not, let me know what is correct. Please reply directly to my e-mail address, not to the mail group. If you want to know the results, tell me: I'll be willing to send out a summary. (If you respond but don't want your information included, please state this.) Regarding short bursts and related issues, I am unclear. * Does anyone have a problem if the first burst is short? * Does anyone have a problem if the last burst is short? * Does anyone have a problem if the first burst is not short? * Does anyone have a problem if the last burst is not short? * Does anyone have a problem if the D2 area starts in the first burst? * Does anyone have a problem if the D2 area starts in the second burst? * Does anyone send a nonzero D2 offset for IP? Regarding the D2-area, I think * All send, and can receive, LLC and SNAP. Regarding the D1-area and HIPPI-LE header for IP, I think * ARP is not used or supported by many, and not required. * Most implementations use tables in place of ARP. * For IP, receivers ignore the HIPPI-LE header. * For IP, senders can fill in the destination IEEE address. * For IP, senders fill in no other fields of the HIPPI-LE header. * For IP, a D1 area with HIPPI-LE header is sent in each packet. * A D1-area with no fill is accepted (3 64-bit words) * A D1-area with no fill is generated by at least some, Regarding HIPPI addressing in the I-field, I think * All implementations can receive when logical HIPPI addresses are used. * All implementations can send using logical HIPPI addresses. Regarding HIPPI connections, I think * Within reason, multiple IP packets can be received on a connection. * Within a connection, distinct kinds of traffic are not sent. (E.g., all IP, all IPI, or all whatever would be sent.) Thanks. Lansing Sloan, LLNL ljsloan@llnl.gov