LSL Wiki : XMLRPCDiscussion

XML-RPC is working, but with the restriction that the script cannot initiate a remote procedure call - it can only answer them. This means, you cannot use XML-RPC to communicate directly between objects. According to a posting from Mark Linden, this will remain the case until they have figured out a way to let LSL scripts do XML-RPCalls without having malicious scripters abuse the feature.

Ezhar thinks they could probably safely enable it for connecting to their own hosts, not the rest of the net, to allow object-object communication within SL. -- Ezhar

Chris thinks that if LindenLab wants to call the functionality they give us "XML-RPC", they should give us a full implementation, not some crippled parts of a full implementation. Malicious scripters? Please post the scenario where fully implementing XML-RPC will somehow cause grief.

Several: Portscanning using XML-RPC requests. Denial of service attacks against any kind of TCP-based service (webservers, mailservers etc.). Any of these things will cause a) people to get very angry at LL and b) people to block LLs IP range. So lets say someone from SL will rez a few hundred objects that each send XML-RPC requests as fast as they can to a webserver hosted by the british telecom. Webserver craps out, BT people get pissed, probably tell LL to stop it immediately and meanwhile block LLs IP range to protect themselves. Result: Nobody using BT as their internet provider can log into SL anymore, LL facing pissed BT users, and possibly legal trouble. Banning the offender from SL will not influence the outcome of this - it's too late, the damage is done. Worse yet, if this happens two or three times (not unlikely), LL can easily loose their credibility on the net and end up having many more providers block their IP range as preventive measure. -- EzharFairlight

I stand corrected. --Chris

Proposed solution to this problem: Check the servers response for the correct message-id:

If the server is not responding/timing out, responding with an invalid XML-RPC reply or responding with the wrong/missing message id (from llSendRemoteData) an error should be returned to the script that initiated the XML-RPCall. The next XML-RPCall from the same script would execute only after a delay (10 - 30 seconds is not unreasonable). Such an error could be sent to the script via a remote_data event with a new constant, for example REMOTE_DATA_ERROR - this way the script can handle an error without delay and then retry the call (and only now the delay kicks in, preventing flooding a service with requests).

In the case of a well-formed reply with the correct message-id, the next call would not be delayed. This would ensure that a server unaware of llRemoteData XML-RPCalls could not be flooded with requests while a server that handles them and returns the correct message-id (and thus can be expected to be a valid target for such requests) can still communicate with the script without delay.

This solution would prevent the abuse of LSL's XML-RPC client as denial of service mechanism, with the exception of attacks on valid XML-RPC/llRemoteData servers. A fairly acceptable risk that could be further lowered by including the scripts owners UUID in each XML-RPC call and thus making tracing the offender very easy.

-- EzharFairlight

maybe keep track of ip's of XML-RPC/llRemoteData servers and delay any request that is sent to close to another request; have each ip have a 10 request buffer, and if the buffer fills, throw an error. Then any time data comes back from that server send 2 requests from the buffer if they are present ignoring delay period but not ignoring some global cap. That way you could never exceed the capacity of the server or flood a server that doesn't send responces. Also would need the sims to tell each other which ip's they were hosing so someone doesn't leverage all 210+ sims resources (maybe a good idea to do the same thing for email). -- BW

Here's Testing Bard's opinion on the subject:

"This problem has already been addressed and solved by Macromedia.

To prevent this same security issue and potential absue, Flash Player 7 changed its security model to only allow TCP and HTTP connections back to the exact address (host.domain) serving the application (SWF). -- Except -- If you specify a host/domain outside the serving address, it will check for a security policy file, crossdomain.xml in the website root.

e.g.

http://foreign.host.domain/crossdomain.xml

<?xml version="1.0"?> <!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"> <cross-domain-policy> <allow-access-from domain="*.secondlife.com" /> </cross-domain-policy>

Before an XML-RPC connection is opened to a foreign server, the policy file on the corresponding web server is checked. That policy can then be cached by SL for 1 hour, 1 day, or whatever they decide. Therefore there is NO DoS potential for any site unless they explicitly allow XML-RPC connections from SecondLife.

For servers that do allow connections from SL, it is unlikely anyone but the server-admin & script author would know the address, unless they make it available for public use. The security file could also be extended to only allow connections from objects owned by a particular owner, or particular sim.

<cross-domain-policy> <allow-access-from domain="sim05.secondlife.com" /> <allow-access-from domain="EzharFairlight.secondlife.com" /> </cross-domain-policy>

The other piece prevent denial-of-service attacks (against another site or the SL sim server resources itself) is to simply limit the number of concurrent XML-RPC calls a given object can make. Also, have a finite XML-RPC object pool on each server limiting the number of concurrent XML-RPC calls the entire sim can make, insuring the integrity of server resources.

Polling the servers from outside poses a much higher and unmanageable server load risk than does initiate connections in-world. " [Testing Bard]

Here's Chris' responce to Testing Bard's suggestion:

"I would love to see outbound RPC calls enabled, and this seems like a plausable solution. The only problem I have with it is how it can be actually implemented.

Ideally, to request security policy information, I'd like the linden server itself to make an XML-RPC request to the destination domain.

Something like, calling a remote function named "isRpcConnectionAvailableFor" with a String argument containing the hostname of secondlife's XML-RPC server ("xmlrpc.secondlife.com" or something like that. The Linden domain and the function the Linden server calls to check its permission have to be known published values). isRpcConnectionAvailableFor will then return a String (or int value) containing "true" or "false" (or 1 or 0, if you go with ints). If the xml-rpc destination address doesn't recognize (or doesnt respond to) "isRpcConnectionFor", then the Linden server would assume the responce is "false". The Linden server would then store the destination domain's responce and purge it every x amount of <minutes, seconds, hours, days>.

That way, people on the Linden end and us SL scripters dont have to jump through hoops or make huge modifications to implement this secure system." [Christopher Omega]

Here's Xugu's thoughts:

"Would it be possible for the system to block outgoing requests to hosts from which it has not yet received XML-RPC requests? Ideally, have a specific command hosts could send to tell the Linden severs that they're willing to accept XML-RPC requests?" [Xugu Madison]

Grim's thoughts

I'm pretty much against Strife's recommendation, and even Testing Bard's solution has some problems. Xugu's idea still wouldn't prevent DoS attacks against VALID XML-RPC servers that are intended for use with SL.

I'm working on setting up a remote database site for storage and retrieval of data for use in SL objects. Problem is, I need ways to ensure that only "subscribers" are accessing the database. I think I could work around Testing Bard's solution if it involved the object-owner level security. It would really suck if somebody got pissed off and DoS?'ed my data server, and I had no way to lock out that user's requests without locking out ALL of my users in the process.

I don't want it throttled or capped at all, especially globally. Sure, a global cap may work if I have 10 subscribers, but what happens when I have 100? 1000? [GrimLupis]

Grim, where's Strife's recommendation? What about my idea? :) -Chris

Umm, I don't remember what that comment was in response to. Oops. The potential problem with your idea, Chris, is the timeout period. What happens if there's an internet hiccup, or I'm rebooting my server, and a request comes in? How long will it be before my address will be cleared from the blacklist? And we have to keep in mind the side-effects that we'll get if there's a minor glitch. How hard is it going to be to code fault-tolerance into our little 16k scripts? Also, I'm keen on the idea of being able to allow/deny access to services based on user name/key. Otherwise there's no way for me to ban a bonehead without hurting my legit users in the process. [GrimLupis]

See my comment about Bard's "username.secondlight.com" (or firstname.lastname.secondlight.com)... that would solve the users problem... just add the users to the permissions file when they sign up. [ResunaOddfellow]

They did a nice job making the order of the integer and string values inconsistent in the various functions too ;-p

Here are some Secondlife.com forum posts about XML-RPC:

Title	Description
XML-RPC Implementations	A post about XML-RPC clients in different programming languages.
Not another XML-RPC Thread!?!	A post discussing the crippling of XML-RPC and possible solutions.

XML-RPC

22-07-2004 Questions?!?! I don't know this stuff. Do XML/RPC, SOAP and CORBA all do essentially the same thing? Is this a subset or a superset of what is accomplished by COM, COM+ and DCOM on Microsoft Machines but this stuff is cross platform? Am I close or am I way behind and way off here? Could someone comment on the differences and similarities of these technologies for ignorant people like me? Also, which technologies are taking root now and which are being scrapped? What should I spend my precisous time learning and what is no longer worth the trouble? Thank you everyone. -- TinkerThetan

I just posted the following into the Feature Suggestions forum; I'll copy it here also.

One possible way to avoid [spamming external sites] would be to have a potential out-calling site register itself with SL, which would then allow scripts to make XML-RPC calls to that site. For example:

My fictitious site, neorebus.com, sends an XML-RPC request to second life, using a specific "registration" channel (e.g. 01234567-baad-c0de-d00d-0123456789ab). The IntValue of the request would be the number of seconds for which the registration is valid, possibly with a maximum (say 86400 seconds, 1 day). The StringValue of the request would be a security code which would be used to allow only authorized objects to talk to the XML-RPC server.

The registration process would record the expiration time of the registration, the security code, and the IP address from which the registration request was received.

After the site is registered, objects within SL could start sending XML-RPC requests to the site. The "channel" of the llSendRemoteData call would have to be the security code listed in the registration, and the hostname portion of the URL for "dest" would have to resolve to the same IP address that was recorded for the registration; if either of these requirements is not met, the XML-RPC request would fail [silently or otherwise].

By restricting access to sites that have already registered themselves with SL, you prevent random DoS attacks, and by including a "security code", you prevent unauthorized access to private XML-RPC servers. (public servers could use a null security code to allow general access).

This was a quick thought this morning, and so there are quite possibly security flaws in this model that I haven't considered; if you think of any, please discuss them here

- Neo

HuntingHare was once a network admin (for 8 years) in his first life. Must point a few things out:

1: Places that create XML-RPC resources are already vulnerable to XML-RPC DoS. How they deal with that is their own site-specific policy.
2: Linden Labs acts as an internet service provider in this case -- they can't be responsible for what their users do with the tools, as long as they're responsive to complaints. Since they have positive economic user identification (including a credit card!) for their players, they can be responsive. (block certain credit cards from using XML-RPC, as an example, since there's a finite universe of credit cards -- and LL can certainly log all usage of the XML-RPC function, and make the number of XML-RPC connections per hour a world-wide shared pool, limiting it by the number of primitives that the script creator has, or by the number of instantiated objects that were created by the script creator in the event of a widely-merchandized script, or by an XML-RPC tax for the script creator, or...)
3: Authentication mechanisms are properly placed on the resource itself, not mechanisms for accessing the resource. (I can understand the concept of wanting to minimize the abuse of a 'proxy' system -- but sheesh, this is supposed to be a functional economy, and it would be a good thing to help reduce the amount of money in the world, if they need it.)

The proper way to look at this isn't as a technical problem, but as an economic problem. XML-RPC requests are a precious resource -- and it's in LL's (and the players') best interests to reduce the abuse of them, else they become less precious as more sites block SL's servers.

Since users of XML-RPC already have to authenticate to the XML-RPC servers they're trying to use, forcing a proxy authentication system (or any kind of system that increases the load on LL or the XML-RPC site owner) just doesn't work well. Putting a L$ pricetag on a certain number of XML-RPC requests would be a start, but I'm not sure that that would be useful either. Minimizing the number of objects per owner that can be accessing a certain data source at a time would be another good thing. (though why LL doesn't implement SOAP requests as well -- for compatibility with such things as Amazon and Google -- is beyond me... unless I just answered my own question.)

Anyway. I've been up for about 15 hours, and I'm heavily caffeinated, so these are just some rambling thoughts -- but these are thoughts designed to help put the problem in the correct domain, instead of a heavy-overhead technical concept. (I must admit, I would very much like to be able to post to my blog from within SL. Write up a notecard, ship it off to my custom blogpost script, and make it easier to do things in-world.)

How about simply proxying the XML-RPC calls through the client? That way only that client is affected if they try to DOS another server.

- Noodles

I, OrangeE, concurr with HuntingHare probably because of having a similar background in network administration. I've been considering making data in the real world represented in SL. For example, I am working on a 'mood ball' that could change colors based on changes in financial markets or other parameters. I was working on making a nice script that would not make RPC requests unless an avi was nearby, but because I can only do XML responses I'll have to push data into the ball from the outside world frequently without knowing if it's worth updating or not. I suppose my point is that without a working llSendRemoteData Second Life is creating an inbound-spam problem of lots of inbound RPC requests for objects that do not need updates.

BTW, can't users already send E-Mail out through the system? Has there been any problem there with spamming?