Serial port access?

[Tsaukpaetra]

I've been puzzling over this for quite a while, since documentation on the internet usually states things like "it should work out of the box as long as it's supported."

I've recently acquired a nice, well supported Fax Modem by US Robotics. Modem works when plugged into a Windows 7 PC just fine, but I want it attached to my server to be run by Windows 2012 as a Fax Server. There is a single COM1 serial port on my supermicro motherboard, with an unconnected header for a COM2. So I plugged the modem into COM1, and this should (theoretically) be the end of my hardware configuration.

I've read from various places that I should use /dev/cuau0 or similar to connect to the modem, but sadly it seems I'm out of my league, as even though cuau0 to cuau2 exist, none of them seem to go to the modem at all, and I don't really know how to tell.

Note that the Serial Port Address drop-down in the system->Advanced page states that I have three ports as well, 0x2f8, 0x3e8, and 0x3f8.

Once I can confirm proper operation in FreeNAS, I will attach it to a VirtualBox VM running Server 2012, which should finally be able to detect it and install it like normal. But that's later since I can't even get a terminal into the com port anyways.

Where can I go from here?

 

[cyberjock]

Umm. I'm not 100% sure what your goal is, but if your goal is to install the Fax Modem on the FreeNAS server and use it, you're going to be disappointed. FreeNAS has no provisions for setting up and using Modems of any kind, so I'm thinking the answer is "not supported". If you aren't trying to connect the Fax Modem directly to the FreeNAS server please elaborate.

 

[Tsaukpaetra]

I'm effectively piping the serial port into a VM. The only reason I mention testing it in FreeNAS first is because I have no idea which /dev entry of whatever to use.
Testing a modem typically involves opening a serial console to the correct port and doing a few simple commands, but I just can't figure out how to do this, and the trial-and-error method just doesn't seem to be helping me (as the session just locks up whenever I try to access any ports).

 

[jgreco]

There isn't that much you can do with it, but you ought to be able to chat with the modem. This is not a stress-free process for someone who didn't grow up in the era of modems, however.

First you've really got to make sure your modem is even willing to talk to you. Depending on the model, there are probably at least a few DIP switches on the unit, so make sure you have:

1) ATA command enabled (and/or dumb mode disabled)
2) Use stored setttings instead of factory defaults
3) DTR mode enabled
4) Carrier detect normal/enabled
5) Enable verbose result codes
6) Command echo enabled
7) Fixed speed set at 9600

You may not have some/any of these, but for the ones you do, these will make your life easier as someone trying to talk to the modem.

From a prompt, then. You're missing the utilities that'd make it easiest to do, like kermit, but fortunately FreeNAS at least includes "cu".

# cu -l /dev/cuau0 -s 9600
Connected.

At this point you're talking to the modem but it may need some more rejiggery, so hit <RETURN> and then a tilde followed by control-Z. You should have just suspended your cu session and returned to prompt.

Then type

# stty -crtscts clocal < /dev/cuau0
# fg

which returns you into the cu session after telling the UNIX tty layer to ignore handshaking.

Try typing "ATE1Q0" and see if it says "OK". If it does - success.

 

[Tsaukpaetra]

There isn't that much you can do with it, but you ought to be able to chat with the modem. This is not a stress-free process for someone who didn't grow up in the era of modems, however.
First you've really got to make sure your modem is even willing to talk to you. Depending on the model, there are probably at least a few DIP switches on the unit, so make sure you have:
You may not have some/any of these, but for the ones you do, these will make your life easier as someone trying to talk to the modem.

From a prompt, then. You're missing the utilities that'd make it easiest to do, like kermit, but fortunately FreeNAS at least includes "cu".

# cu -l /dev/cuau0 -s 9600
Connected.

At this point you're talking to the modem but it may need some more rejiggery, so hit <RETURN> and then a tilde followed by control-Z. You should have just suspended your cu session and returned to prompt.

Then type

# stty -crtscts clocal < /dev/cuau0
# fg

which returns you into the cu session after telling the UNIX tty layer to ignore handshaking.

Try typing "ATE1Q0" and see if it says "OK". If it does - success.

I did indeed grow up with modems, but never had to try talking to one with Unix/BSD/Anything other than DOS or Windows. ;)

I'll double-check the DIPs, and thanks for the commands. I had gotten to the point of cu -l /dev/cuau0, where it would claim "Connected", but nothing after would ever come back (and for all intents and purposes the was no IO going on at all).
The fun part will be, once I can confirm I have it set up correctly, how to patch it into the VM, since VirtualBox itself has exceedingly sparse configuration in that regard (It amounts to "Specify device path", and that's pretty much it).

 

[Tsaukpaetra]

The DIP switches may have been the key, using the cu and stty commands now gives feedback!
Stupid Windows installed it as a Serial Mouse though. :/ Now I gotta figure out where they hid that setting in 2012...

Funny thing now, When I ask Windows to query the modem for standard settings (Diagnostics tab on the Modem Properties), it apparently sends out the commands OK, but the response doesn't seem to come back to the VM. Evidence in point, if I do cat /dev/cuau0 and ask for a query, the cat command gets the "OK" response, when I would think that the cat command should get some kind of "access denied" message.
Might just be all the messing around I've been doing, I'll reboot and see if that helps anything.

 

[jgreco]

when I would think that the cat command should get some kind of "access denied" message.

And why exactly would you think that?

UNIX doesn't come with a lot of safety belts. Preventing users from doing things that might be stupid also prevents them from doing things that are clever.

 

[Tsaukpaetra]

And why exactly would you think that?

UNIX doesn't come with a lot of safety belts. Preventing users from doing things that might be stupid also prevents them from doing things that are clever.

Yes, I continuously find myself in the trap of applying Windows concepts to Unix machines. :/ For example, on the Win2012 side, no application can open a COM port if another app already has access.

In this case I expected similar protection, in the event that some session should not get hijacked by another process.
I get error messages when I try to open /dev/ttyu0 for example (device busy), and I assume this is because VirtualBox is holding the cuau0 side open.

I think I'll play with the dip switches a bit more, since some portions of the communication are indeed coming through now, but only about 70% of the time, which I think might relate to the clear-to-send and request-to-send being disabled. There should be something in the DIPs that should set that, but I'm not in front of it to check at the moment.

 

[jgreco]

You can certainly lock a resource exclusively in UNIX. It just doesn't happen magically/automatically/etc especially if you're using something like "cat". Cooperative management of a device isn't horribly unusual.

Being a USR product, it probably supports AT&H0 and friends to display help pages, which are a fine way to test your signalling... just look for obvious corruptions.

 

[Tsaukpaetra]

It's weird though, how inconsistent it is. For example, during the best of times, I can get the following output when doing ATI0 through ATI7 (apparently AT&H0 just returns "OK").

Code:

ERROR

OK

OK

5601

OK

11D4

OK

OK

OK

U.S. Robotics 56K FAX EXT V4.7.31

OK

U.S. Robotics 56K FAX EXT Settings...

   B0  E0  F1  M1  Q0  V1  X1  Y0
   BAUD=9600  PARITY=N  WORDLEN=8
   DIAL=TONE    ON HOOK   CID=0

   &A1  &B0  &C1  &D2  &G0  &H1  &I0  &K1
   &M4  &N0  &P0  &R1  &S0  &T0  &U0  &Y1

   S00=001  S01=000  S02=043  S03=013  S04=010  S05=008  S06=002
   S07=060  S08=002  S09=006  S10=014  S11=070  S12=050  S13=000
   S15=000  S16=000  S18=000  S19=000  S21=010  S22=017  S23=019
   S25=005  S27=000  S28=008  S29=020  S30=000  S31=128  S32=002
   S33=000  S34=000  S35=000  S36=014  S38=000  S39=000  S40=001
   S41=000  S42=000

   LAST DIALED #:

OK

U.S. Robotics 56K FAX EXT NVRAM Settings...

 Template Y0

   DIAL=TONE   B0  F1  M1  X4
   BAUD=38400  PARITY=N  WORDLEN=8

   &A3  &B1  &G0  &H1  &I0  &K1  &M4  &N0
   &P0  &R2  &S0  &T5  &U0  &Y1

   S00=001  S02=043  S03=013  S04=010  S05=008  S06=002  S07=060
   S08=002  S09=006  S10=014  S11=070  S12=050  S13=000  S15=000
   S19=000  S21=010  S22=017  S23=019  S25=005  S27=000  S28=008
   S29=020  S30=000  S31=128  S32=002  S33=000  S34=000  S35=000
   S36=014  S38=000  S39=000  S40=001  S41=000  S42=000


Strike a key when ready . . .


 Template Y1

   DIAL=TONE   B0  F1  M1  X4
   BAUD=38400  PARITY=N  WORDLEN=8

   &A3  &B1  &G0  &H2  &I2  &K1  &M4  &N0
   &P0  &R1  &S0  &T5  &U0  &Y1

   S00=001  S02=043  S03=013  S04=010  S05=008  S06=002  S07=060
   S08=002  S09=006  S10=014  S11=070  S12=050  S13=000  S15=000
   S19=000  S21=010  S22=017  S23=019  S25=005  S27=000  S28=008
   S29=020  S30=000  S31=128  S32=002  S33=000  S34=000  S35=000
   S36=014  S38=000  S39=000  S40=001  S41=000  S42=000


   STORED PHONE #0:
                #1:
                #2:
                #3:

OK

U.S. Robotics 56K FAX EXT Link Diagnostics...

Chars sent                    0      Chars Received                0
Chars lost                    0
Octets sent                   0      Octets Received               0
Blocks sent                   0      Blocks Received               0
Blocks resent                 0

Retrains Requested            0      Retrains Granted              0
Line Reversals                0      Blers                         0
Link Timeouts                 0      Link Naks                     0

Data Compression       NONE
Equalization           Long
Fallback               Enabled
Last Call              00:00:00

Disconnect Reason is Keypress Abort

OK

ERROR

Configuration Profile...

Product type           US/Canada External
Product ID:            00568600
Options                V32bis,V.34+,x2,V.90
Fax Options            Class 1/Class 2.0
Line Options           Caller ID, Distinctive Ring
Clock Freq             92.0Mhz
EPROM                  256k
RAM                    32k

FLASH date             2/12/98
FLASH rev              4.7.31

DSP date               2/12/98
DSP rev                4.7.31

OK

When doing this through Putty all seems well. It seems that having cat listening to it at the same time somewhat divides the output, so obviously attempting to watch the modem in this manner is a no-no.

I guess the alternative is that the modem really isn't working quite right (It says that it's answered the phone, but that shouldn't be right since there's still a dialtone...).
I also attached the Windows Modem log so far in my adventures.

 

[Tsaukpaetra]

Yeah, I think it's the modem that's busted.
The modem can apparently mostly give and get information, but very unreliably. I'm going to try another modem to double-check it's not that. It's weird but maybe there's some kind of compatibility issue?

Thanks for all the help so far!

 

[jgreco]

When doing this through Putty all seems well. It seems that having cat listening to it at the same time somewhat divides the output, so obviously attempting to watch the modem in this manner is a no-no.

I guess the alternative is that the modem really isn't working quite right (It says that it's answered the phone, but that shouldn't be right since there's still a dialtone...).
I also attached the Windows Modem log so far in my adventures.

Yes, the correct method for monitoring would be something involving watch, but I'm too lazy to go read the manpage. It gives you a software version of a hardware tap, and it's better than a hardware tap in many ways.

That modem looks like it is some bastard revision of a Sportster. I've got at least one Sportster in inventory somewhere so maybe I can go dig it out.

 

[Tsaukpaetra]

Yeah, I picked up a new modem. I'm pretty convinced that it's a VirtualBox issue, as I can communicate just fine on the FreeNas side, but once inside the Guest, nothing goes through quite right.

 

[Tsaukpaetra]

Or maybe not? Seems the kernel's serial port settings may be interfering after all? I was messing with the stty command and accidentally managed to get it to 94% integrity (I think the device was overflowing the buffer? Simple AT commands worked but as soon as data connections were attempted things went downhill).
Unfortunately I have no idea what the used settings were, so I can't replicate it (somehow the kernel panicked on accident while disconnecting a session).