Keepalives.

Posted by Neva on Mon 29 Aug 2005 12:41 AM — 24 posts, 83,626 views.

USA #0
I realize that one can do this with timers, but the problem with timers is that it reports to the game that you're active, even if you're AFK while the timer is running.

I'm not entirely familiar with the telnet protocol, but is there a way to add something to the client which would tell the client to send a packet periodically that wouldn't be recognized as a 'command' by a TinyMUSH server, but would keep the connection active so that NAT routers wouldn't drop it?

Just curious.
USA #1
MUDs usually don't implement the full telnet protocol. Even if telnet does have a special keep-alive directive (using an escape character, for instance) then the MUD still has to support and hence process it. After all, what you're sending the MUD is a stream of bytes; the MUD then decides what to do with this stream of bytes. E.g. interpret it as telnet directives, or interpret it as characters, etc.

I think that unless you find a way to do this on a TCP/IP level, what you're asking is impossible. And apparently (see http://home.student.uu.se/j/jolo4453/projekt/tcpip1/tcp_keep.htm) the keep-alive feature of TCP/IP is not a part of the specification. It would only work if both the client and server implementations support it.

Why is it such a bad thing to let the MUD know you're active? Is it to maintain AFK status or something like that?
USA #2
All I know, offhand, is that the TinyMUX server does, in fact, implement a server-side method for doing this, and nothing displays on the screen, which indicates that there has to be some way to send information of this sort. And PennMUSH supposedly has a way to set the *socket* so that it won't do this. So it seems like there ought to be a possibility to handle it on the client side.
USA #3
It sounds like the TinyMUX server implements its own version of keepalives, but the MUD still knows that you sent the information. Whether or not that affects e.g. AFK status or whatever it is you don't want it to affect is an open question. As for PennMUSH, they probably use the TCP/IP keepalive extension I mentioned, but since that's non-standard I'm not sure how good of an idea it is.
USA #4
All I'm saying is, some software projects have managed to do it. MUSHclient includes all kinds of things which are by no means found in all servers, and this is becoming a prevalent enough problem that I don't see what the problem would be in Nick considering implementing it.

As far as the why? A by-no-means-complete list:

* Some games on slower machines don't want resources being tied up by users who aren't really there. Some just don't want the WHO cluttered. For whatever reason, most MUSH/MUX-type games have idle timeouts set, and the timers used to prevent the (very short) connection timeouts of NAT routers also prevent those from working properly.
* Some games maintain activity counters which are used to determine things like eligibility for important positions. The 'idle' time is supposed to symbolize how long it's been since you last did something on the game, not how long since your client last performed some command for you, but that's often what it ends up being.
* Lots of players on the WHO who look like they're active and possibly available for RP but aren't make a game look bad. On MUSHes, at least, you don't get somebody's idle message unless you page them directly.
* If someone with a timer gets called away unexpectedly and forgets to sign off, they can be left online for days or longer if they're away and their computer remains turned on, during which time, again, there's no real way to tell if they're present or not without just paging them repeatedly and seeing if they reply.

Maybe it doesn't matter, on a combat-oriented MUD, but not all of us are playing those sorts of games. My games are much more socially-centered, and being aware of who's present, who isn't, and how much time they're actually spending on the game is fairly important.
USA #5
I don't think you quite understand what I said. There are two layers of communication between a MUD client and a MUD server: (MUD being used in the general sense of multi-user text-based game)

Layer 1- TCP/IP. This is controlled by the operating systems (well, almost, but let's pass on that) and is a means of getting raw data from one end of the connection to the other.

Layer 2- Text over TCP/IP. This is how the client and server actually communicate. As I mentioned, this is just a stream of bytes that the server decides to interpret one way or another.

The server does not see what happens in layer 1. That is because the entire layer is abstracted away from the program. The server does, however, see everything that happens in layer 2, because the server is where these bytes are sent and the server is the one that processes them.

Unless the server implements some special handling in layer 2 of 'keepalives', such that it basically throws them away and does not consider them for the idle timer, what you ask is impossible in layer 2 of communication. Even when it is possible, it is specific to one server code base in particular that handles this particular keep-alive.

Layer 1 (TCP/IP), then, is your only hope. As I mentioned, TCP/IP does appear to have a non-standard keep-alive extension. This has nothing to do with combat MUDs, social MUSHes, MUSHclient, the server, etc.; in other words, it has nothing to do with what MUSHclient itself supports or not, and it has nothing to do with what the server supports.

It depends on the TCP/IP implementations on both the client and server machines. If both support the keepalive extension, then that's great and there are no problems. If however neither support the extension, then nothing will happen either. But if only one supports the extension and starts sending keepalives, what is going to happen? The best case is that the other side will ignore the keepalives; the worst case is that it'll interfere with the connection data and the non-supporting side will treat the keepalives as actual data, which is very wrong.

I hope this clears up what is going on from a programmer's perspective. From past experience Nick is (IMHO rightly so) not fond of non-standard protocol features, so it's relatively unlikely that he'll implement the TCP/IP solution. Your best bet is to talk to your game administrators and ask them if their code supports any kind of keepalives that don't interfere with their idle counter.
USA #6
It's not a matter of the game administrators. It's a matter of the servers, which unlike MUDs, are not coded by the people who run them in any way, shape, or form, in all but very rare cases. Some games run on servers which are no longer even supported, so the chances of new features being added? Nil.

If a non-standard extension, which can be enabled optionally by the user the same way they can choose to enable/disable Pueblo/MXP and the like, would solve this problem, then why shouldn't it?

Lots of things in MUSHclient are optional and not supported by all games by any stretch. That doesn't mean they're not good and valued features for the people who use them.

From the standpoint of this timeout? All that needs to happen is that something be transmitted over the connection periodically, as far as I've ever been able to tell. If, as you say, half of this packet is ignored by the server, then it would seem like theoretically one could even send just one now and then where that part was empty and it'd work. Which is again, not something one can currently do. The game itself doesn't need any input, and in fact shouldn't have any input, but the connection needs something sent to stay going.

The differences between MUDs and social/RP games that I noted were about the importance of idle time showing up as idle time. Like I said, maybe that's not a priority for your type of games, but it is for others.
Amended on Mon 29 Aug 2005 06:15 PM by Neva
USA #7
Quote:
If a non-standard extension, which can be enabled optionally by the user the same way they can choose to enable/disable Pueblo/MXP and the like, would solve this problem, then why shouldn't it?
Because this isn't an extension like MXP and Pueblo! Those all happen over layer 2 of the communication. What you're asking for, if you cannot talk to administrators and have them implement specific support for this, needs to happen over layer 1 of the communication.
Quote:
If, as you say, half of this packet is ignored by the server,
I didn't say that. I said that is what you would have to do server-side. Server being the game server, not the machine it's running on.


As I said, the difference between combat-oriented and social games is completely irrelevant here...


There is another possibility, which would be to send an empty packet on TCP/IP. I'm not sure if this is possible, because implementations might not actually send any data if there's nothing to be sent.
USA #8
It is the same, in the sense that if you don't check the box on the client, the client doesn't do anything different than it usually does. Information goes back and forth and nothing changes. If something were added in this sense, if you didn't have the keepalive box checked, nothing would be different. If you didn't want it turned on, it wouldn't affect you at all.

Once again, all the differences between your type of game and mine were with regards to the question YOU ASKED:

Quote:
Why is it such a bad thing to let the MUD know you're active? Is it to maintain AFK status or something like that?


Not with regards to the method under which it could be handled. Which I'm sure there are more than one, and which method is workable and best should be up to Nick, I'd imagine, not anybody else.
USA #9
No, it is NOT the same. It's not just a question of adding a magic checkbox that magically makes this stuff work over a magically powerful internet connection. There are some serious technical issues here, and every time I try to tell you about them you brush them aside as if they were meaningless.

The problem here is quite trivial. You're speaking from a user's perspective and I'm speaking from a technical perspective. I have the impression that either you did not read or you did not register what I said about the two layers of communication. It'd be a lot easier if you worked with me instead of continuously ignoring the problems I bring up. If you want me to rephrase or explain the problems, then just say so. But please don't just brush them aside and repeat yourself.
USA #10
I'm well aware of the two layers of communication. I understand this. I'm not a programmer, but I'm also not stupid, thank you. Proxies aren't magical checkboxes either. But if you don't have them turned on, your connection works just the same way it always did. When it's turned on, the client routes it through a different server as instructed.

I'm not especially concerned with the technical manner of doing this because there's not just one, as it evidenced from the number of other programs, servers and otherwise, which maintain network connections of all sorts in whatever way they see is best. No, it's not trivial, in terms of the time it might need to implement it, but a lot of features in MUSHclient haven't been.

But going into that, one part of the packet is the text it's sending and the other part are the associated instructions with the packet. Again, not a programmer, but not stupid. If the instructions could be sent without text, then this would also serve as activity on the connection and prevent the NAT from timing it out. Or the optional keepalive part could be used, as a part of the connection options which must be set before each connection, and left out when not in use, depending on how that interacts with servers which don't work with it. I don't know that, but to be honest, it doesn't sound like you do, either.

I resent being talked down to in this manner, and I really don't understand why you're so dead set against the developer of this particular software package taking the time to actually evaluate this proposal. You're not the one who has to do the code, so maybe you could leave it up to the guy who does. Nick's implemented a number of things along the way which have to have been at least as complicated as this (if not moreso, imagining the amount of time that's gone into scripting since it first went in), and I hardly think it's impossible.
USA #11
Ksilian, it might be a simple case of the server sending a certain packet, and expecting a response (SO_KEEPALIVE, and expecting an ACK response like). Which is doable in mushclient (with a plugin none the less!).

But honestly, I can't find ANY documentation on this for TinyMush, so we really can't tell. Neva, do you have any links or blips or anything about what is actually going on with these keepalive things?

Edit:
And if there are different implementations for different servers (provided they all work as plugins) then I would think that it'd be best left up to a plugin to do the work. It doesn't require further integration, and there is no standard way of doing it, there will have to be a bunch of options about it (or at least a dropdown list) and we would never really be able to know for sure if we got them all.
Amended on Mon 29 Aug 2005 06:50 PM by Flannel
USA #12
TinyMUSH doesn't have any sort of capabilities for them. (When suggested, the response was basically 'go to hell, that's the client's problem', which makes it interesting that Ksilyan's response is the exact opposite.) TinyMUX has something coded into the server, but that's not doable for most of us, since the average player has no control over what server the game s/he plays uses. PennMUSH has something which uses the aforementioned non-standard keepalive extension. I don't know how well it works, because I don't know of anybody who plays on a Penn-based game right now. (Not that people don't, just not my circle of friends.)

Because the problem of the connection timeouts is on the client's side of things--not actually in the client, but in the client's computer's connection to the computer, and therefore really not the server's problem--it seems like a fairly reasonable thing to see if the client can be made to do something (send an empty packet periodically, use the keepalive extension, hop on one foot and sing, I'm not picky as long as it has the desired effect) which would send data through the connection, making sure the NAT doesn't close it, without the MU* server registering it as a command.

For the time being, the option favored to keep the connection going is a timer with '@@' or some other command which produces no result, but that still tells the server you're there when you aren't. I don't know, because I don't do a lot of scripting anymore, if any of these other options are things that plugins are capable of or not. If so, then terrific, it just seems to be something deep enough into the way the client actually operates that it might not be scriptable.

In which case, it's the sort of thing Nick could look into.
USA #13
Flannel: yes, sending an empty packet is one possibility, as I mentioned. But again, that depends on the TCP/IP library not throwing away empty packets - and it's not unreasonable for a library to do just that. As for SO_KEEPALIVE, it seems that that is non-standard, which can create problems.

Neva, I'm not trying to talk you down. What I am trying to say is that you're not registering what I'm telling you, either that or you're not reading it. Sure, lots of programs implement their own versions of keepalive. But how do you know whether or not they do it on layer 1 or layer 2 of the communication? If they do it on layer 2 - which is quite likely - then, translated to our particular situation, the game server has to be changed. You already stated that is not an option, so . . . that pretty much rules out that entire set of possibilities, doesn't it?

As for this:
Quote:
If the instructions could be sent without text,
I already mentioned that; I also mentioned before (and to Flannel just now) that that solution depends completely on the library not throwing away empty packets. After all, in the vast majority of cases, it is a good thing to not send data if there's nothing to send.

As a final note, what you're asking for is actually trivial. With, of course, the disclaimer of everything I've said... For instance, the TCP/IP keepalive business is just a matter of setting a flag. That's one line of code. But it's a non-standard flag. And that's not non-standard in the sense of MXP. It's a much more fundamental "non-standardness" than that. That is what I feel you have not accepted yet: it really isn't a question of programming difficulty, it is a question of programming feasibility and good practice.
USA #14
[You posted your post as I was writing mine.]

Quote:
which makes it interesting that Ksilyan's response is the exact opposite.
Would you mind elaborating a bit on that? I don't want you putting any words in my mouth... :)
USA #15
And having the specs in the first place. In it's most basic form, it's a plugin with one trigger (and two entities), and actually, that trigger would probably end up being a callback.
But, if we don't know what to look for and send back, we can't help.
USA #16
That's the thing... if the server implements its own version of these keepalives, then the problem is trivial. But Neva stated that it's not acceptable to expect such an implementation nor is is acceptable to ask for it to be implemented, so this whole set of possibilities is basically ruled out.
USA #17
She was saying that her server people said "its the client's problem" when you were saying "its the servers problem".

But, we were talking about two different things. We were talking about how to implement a keepalive system, she just wants her NAT not closing her connection (but there's no support for it on the server side). It really is 'the clients problem' as you 'chose' to play through a router that drops you after a certain amount of time.

Unfortunately, without sending something to the server, there isn't any way to keep your connection alive. And, unless you can do some negotiation (like pretending to be interested in MXP, or whatever else) which doesn't reset your idle time (it may or may not, it would depend on the server I think), then your timer is going to be reset whenever you do it.
USA #18
Actually, have you tried sending a KeepAlive packet?
Hmm, I would think if the server doesn't support it, you would end up closing your connection (maybe?) if done by the OS (as it wouldn't get a response?).
But, if you're sending via your client, it would serve to keep your connection alive (as you would be sending packets) and, your server might not count it as sending something (or it might, but I imagine it wouldn't display it).
It's worth a try, either way.

Documentation:
http://www.dslreports.com/faq/7792

Talks about what keepalive is (I thought it was coming the other way), and how to change your operating system default timeout period, but if TinyMush doesnt support that, I think it would close the connection (not at the NAT, but at the OS, thinking the server had died).
USA #19
It's not that it's not acceptable to ask. I realize you come from a MUD background, so this is different. MUDs have coders who deal with the servers. I *wish* we did, but we don't. We have server maintainers who do everything, and then it gets packaged and the game owners just install it and run it. Many games are running these packages from versions that were last up-to-date sometime pre-2000. So they're not going to be getting new versions with keepalives even if the servers put out those versions.

It'd be preferable, really, if new servers had keepalive systems *and* the client did, to handle both old clients and old servers which don't. If the server can implement it without the client doing anything, though, it stands to reason that the client ought to likewise be able to implement it without server support. If it had to be two-sided, the versions already out there wouldn't work. There's no logic to the claim that it can't be done in the client if the servers--and not just the one that does it with special commands--can manage it on their own.

Ksilyan: http://www.pennmush.org/mail-archive/pennmush/2003-September/001554.html
It's talking about setting things on TCP sockets. It's not just having people send a command or sending some text to the user, which would be layer 2, wouldn't it?

USA #20
You can't do it SOLELY on the clientside, without basically just having a trigger that sends something.
If the server doesn't support it, you'll be considered 'active', unless you find some stuff to send that doesn't make the server think you are. This is akin to making a timer, that sends (whatever), you'll have to play around with what to send per server (and you might get lucky and find something that the server doesn't reset your idle time with).

Responding to a server SO_KEEPALIVE is doable in a plugin, but honestly, I'm not sure if you need to respond at all. The NAT just needs to see some data passing and it won't disconnect.
USA #21
In which case, if you don't know how to work this with a plugin, can we leave this for Nick to look at and see if he thinks it's doable with the client or not? All this talk is very nice and all, but to be honest, he's the only one who can really make the distinction about what he can and cannot accomplish, and I know he's got at least some background with MUSHes to know what they might or might not be able to work wtih.
USA #22
Quote:
It's talking about setting things on TCP sockets. It's not just having people send a command or sending some text to the user, which would be layer 2, wouldn't it?
That's correct, it would be affecting layer 1. The unfortunate part is that these keepalive extensions are precisely that, non-standard extensions to the TCP/IP protocol. It's unclear to me what would happen if one side was using keepalive and the other wasn't. It's possible that simply nothing will happen, or that one side will drop the other (e.g. "You just sent this my way but it doesn't make sense to me, so go away").

Nick doesn't really have much say in the matter since he doesn't control the TCP/IP protocol. What Nick can do is turn on the (non-standard) extension, but as I said before, given past experience he is not eager to use non-standard extensions and personally I would tend to stand behind that.

Allow me to make this distinction again: the fact that we're talking to a MUD or a MUSH or a WhatTheHeck is really quite irrelevant. It doesn't matter if somebody has a background with them. If you want this to happen on the TCP/IP level, it doesn't matter what kind of program is the client and what kind is the server.

If you want to do this on the second layer - the one the client and server see - then the server needs to support such a mechanism. It's trivial to implement. It is, however, as you say, less trivial to get the change distributed to all the various game servers.

Quote:
There's no logic to the claim that it can't be done in the client if the servers--and not just the one that does it with special commands--can manage it on their own.
What's probably going on is that the server is using the non-standard extension. If the client supports the extension as well, it will issue keepalives as expected. However I hesitate to make big claims about these things until I actually see it happening and have a chance to look at it.
USA #23
It's not that I don't know how it would work with a plugin, I just have no documentation about what actually happens.
Once the actual occurances are settled, it really is trivial to turn it into a plugin. Real documentation is needed, regardless of how it is implemented. So, I'll ask again, can you provide any decent documentation as to the protocol/specifications of this keep alive stuff?