Loading…
and others
Each of the network extension points requires special permission from Apple :(
Now that Apple allows anyone to run the code on their own devices, we don't have to publish the app on the App Store.
No, it still requires some entitlements to run on the devices.
Totally, but - the API documentation is hard to piece together and there is no template in Xcode for the extension point. Gonna have to do some reverse engineering.
There's no documentation at all at the moment. The headers of NetworkExtension.framework are public, so we can figure out how to implement the proxy.
I guess we need to subclass NEAppProxyProvider to handle both NEAppProxyTCPFlow and NEAppProxyUDPFlow. And somehow activate the proxy.
Or we can subclass NEPacketTunnelProvider to create a VPN tunnel that handles NEPacketTunnelFlow.
Totally. We need to find the extension point identifier, too. Cisco and OpenVPN need to update their apps...
I guess it works just like an app that controls IPSec VPN settings. Before calling manager.connection.startVPNTunnelAndReturnError, we should register our own protocol with
[NETunnelProviderManager loadAllFromPreferencesWithCompletionHandler:(void (^)(NSArray<NETunnelProviderManager *> * __nullable managers, NSError * __nullable error))completionHandler]
I'll give it a try when I have time.
I'm going to wait for the single WWDC session before diving in
NEAppProxyProvider is actually per-app exclusive. Good news is we can use NEPacketTunnelProvider to create global VPN services.
I'm writing to Apple to see if we can get permission for the API.
Have you made any progress on packet tunnel?
Still no reply from Apple.
I'm writing to Apple to see if we can get permission for the API.
So does this mean only those who have grant permissions from Apple can develop global proxy apps?
I'm afraid yes.
I'm afraid yes.
Sad but reasonable. Good luck with SS. 
The NEAppProxyProvider API only require a MDM deployed app. That can be "simulated" as described in the video.
There're actually templates for Xcode. You need to install them from
/System/Library/Frameworks/NetworkExtension.framework/Versions/A/Resources/NEProviderTargetTemplates.pkg
But I have not found the way of how to activate a vpn. As there's no shared instance for NETunnelProviderManager I think we need to create a new one.
[NETunnelProviderManager loadAllFromPreferencesWithCompletionHandler:^(NSArray<NETunnelProviderManager *> * __nullable managers, NSError * __nullable error) {
if (managers.count <= 0){
NETunnelProviderProtocol *protocol = [[NETunnelProviderProtocol alloc] init];
protocol.providerConfiguration = @{ @"some parameter" : @"some value" };
protocol.providerBundleIdentifier = @"com.example.vpn.vpntunnel";
NETunnelProviderManager *manager = [[NETunnelProviderManager alloc] init];
[manager setProtocol:protocol];
[manager setLocalizedDescription:@"My VPN"];
[manager setOnDemandEnabled:NO];
[manager setEnabled:YES];
[manager loadFromPreferencesWithCompletionHandler:^(NSError * __nullable error) {
NSLog(@"%@", error);
}];
}
}];
On the line NETunnelProviderManager *manager = [[NETunnelProviderManager alloc] init];, the following message appears in the console app:
6/27/15 5:31:13.845 PM VPNOSX[1403]: Application does not have the required entitlements.
It doesn't say which entitlements and there's no any documentation about it.
I want to try this api on MAC OS 10.11. I understand the reason why I need to ask apple for some permission to publish the app with this api to app store, but I can't believe that I have to ask them for permission to run this api on my development machine.
Sorry, that's a little bit off topic, but that is the only thread that I found in the internet so far.
Yes. You need to send an email to Apple to get the entitlements. And I'm waiting for their reply.
Any luck on this yet? I am looking at NEAppProxyProvider for a project for a client. I think I do understand some things, but can't be sure till I can run it on the device.
Maybe this demo is helpful to this problem?
let newManager = NETunnelProviderManager()
You'll get a warning complaining about missing entitlements when you execute this line of code.
In README.md it says:
The NEProvider family of APIs require the following entitlement:
<key>com.apple.developer.networking.networkextension</key>
<array>
<string>packet-tunnel-provider</string>
<string>app-proxy-provider</string>
<string>content-filter-provider</string>
</array>
</plist>
The SimpleTunnel.app and the provider extensions will not run if they are not code signed with this entitlement.
You can request this entitlement by sending an email to networkextension@apple.com.
If you try to compile the app with this entitlement, your app will be killed by taskgated daemon. If you like to move com.apple.taskgated.plist from /System/Library/LaunchDaemons with root permission, you will get a nice response:
sudo mv com.apple.taskgated-helper.plist ~
mv: rename com.apple.taskgated-helper.plist to /Users/alex/com.apple.taskgated-helper.plist: Operation not permitted
means that you are not admin now, you are nothing and you are in sandbox:
7/9/15 12:37:27.138 PM sandboxd[113]: ([3711]) mv(3711) System Policy: deny file-write-unlink /System/Library/LaunchDaemons/com.apple.taskgated-helper.plist
We just received the entitlements.
Got the entitlements, too.
Did you apply as an individual or as a company?
I didn't dare filling the form because it seemed like you had to apply as a company.
I applied as an open source organization. I explained a bit about this project in the Company name and address field.
Update:
Now I can get a virtual tun device running and route packets through UDP. While I find it a little hard to debug as I can't attach to the extension.
Now I have ShadowVPN fully working on an iPad. The next step is to add UI, etc.
Jul 19 12:17:56 new-iPad ShadowVPN(NetworkExtension)[1242] : MDM must be used to create NEAppProxyProvider configurations
Looks like NEAppProxyProviderManager isn't for us. Thus to implement Shadowsocks for iOS, we need convert it to a VPN. Maybe we have to port tun2socks to iOS.
@linusyang has a working port of tun2socks on iOS. Maybe he can help.
@madeye @clowwindy I thought we could directly use application-layer proxy by NEAppProxyProvider to implement global proxy.. Then the same trick for Android needs to be used on iOS? If so, because iOS does not allow to run a separate process for tun2socks and tun2socks uses its own event loop (directly built on kqueue), we may need to embed shadowsocks client directly into tun2socks. I will work on this if you are not interested. :)
In some old version of Shadowsocks-iOS, I had polipo's main function renamed and called from a background thread. We can do the same to tun2socks and turn it into libtun2socks. Instead of letting tun2socks create the tun device, we'll create and pass the tun fd to its main function.
Since the VPN extension is a hard place for debugging, we may debug libtun2socks on OS X where it's possible to create a tun fd in a normal process. Then the only thing left is to change the project target.
I think I'll begin working on this in mid August after I'm done with ShadowVPN and ChinaDNS (we have to do some DNS hijacking since it's not possible to listen on port 53). According to records iOS 9 will be released around Sep 18.
If you're interested in porting the library, I've created a repo for it.
https://github.com/shadowsocks/tun2socks-iOS
@clowwindy I did some work on this a year ago but I don't really remember how far I got because I eventually hit a wall.
https://github.com/chrisballinger/badvpn/tree/tun2socks-xcodeproj
https://github.com/chrisballinger/badvpn/tree/darwin-utun
Not sure if any of that is useful anymore.
I'm happy to help in any capacity that I can. I want to get Tor working on iOS, which will require the same NEPacketTunnelProvider -> SOCKS bridge.
Correct me if I am wrong, but the parts we need are:
These pieces on their own don't sound awful to implement while looking at existing implementations – or am I misunderstanding the complexity?
@conradev Awesome work! I have been reaching out to other people who have expressed interest in collaborating on an iOS 9 Tor implementation and I'm glad to see you've already made some progress. Sent you an email. :)
@conradev @chrisballinger
Good to see we can join forces on the tun to socks bridge!
@clowwindy It is a good suggestion to construct tun2socks as a library. I'll first try to make one to run on OS X.
@linusyang I've added you to the team. You can push directly to the repo.
@clowwindy Thanks!
@clowwindy @linusyang @conradev @chrisballinger any one having a sample code for NetworkExtension to scan the wifi list? I guess NEHotspotHelper will be used, but i dont know how to implement
Do application that uses Extension Framework can run in background or should i do something specific to run my application in background?How do I run my custom VPN application in background?
I working on getting NEAppProxyProvider working. Not much luck so far. I think it the Proxy has to be configured either through the code or through an MDM config profile. Don't know which one. Is anybody working on this? Would like to compare notes.
Using the extension Framework I can get it working on the idp_ip0 but if I start with en0 it gives an error 49. However if I switch from idp_ip0 -> en0 it works. Any one else see this problem?@clowwindy
@clowwindy, @linusyang, would love your thoughts on this document.
Trying to actually figure out how to implement this library. I don't have the entitlements yet, so it's all I can do while I wait.
Hei, have you been able to run the sample code? I'm getting "Error Domain=NEConfigurationErrorDomain Code=11 "IPC failed" UserInfo={NSLocalizedDescription=IPC failed}" when trying to use NEFilterManager. I created provisioning profiles for each extension + the main app. I expect this to be a signing problem, but I just can't see it. Thanks
Update: ShadowVPN-iOS is now fully functional.
I've created UI, added CHNRoutes and ChinaDNS. You can also see how to deal with network status change (Wi-Fi/4G). You can try to compile and run it if you have the entitlements from Apple.
The bad news is, I found that UDP traffic often gets blocked very quickly (~10 minutes) in 4G network in China. After toggling Airplane mode twice to get a new 4G IP, the server received data again. But after a while it got blocked again.
So I'm coming back to working on Shadowsocks again. It seems that we can't get the file descriptor from NEPacketTunnelProvider API. As @conradev proposed, we can pass the data directly from and to PacketPassInterface and PacketRecvInterface and let tun2socks reassemble the TCP flow. For UDP, I think it's really easy to write our own forwarder rather that using tun2socks.
@tredds are you running it on device? I was getting this error when was trying to run it in ios simulator.
@angelovAlex Indeed I was running on simulator. I'll give it a try on device.
@clowwindy on China Mobile or China Unicom 4G network?
@conradev @linusyang
tun2socks is based on lwIP, which is a pure C, full stack TCP/IP protocol implementation.
We can just build our own Socks adapter from lwIP and GCD, without tun2socks's event loop.
@clowwindy @linusyang @conradev
Are you saying rewrite tun2socks from scratch, or just rip out portions of it for a partial rewrite? Either way we will also be working on this problem for our iOS 9 Tor VPN effort, and would love to work together where possible.
@angelovAlex It worked. Thanks man!
@chrisballinger
Actually tun2socks didn't do much work. It's lwip that's doing the magic.
So I'm thinking about writing a tun2shadowsocks directly based on lwip and GCD, it would not only be easier but save a lot of sockets and RAM. (We can save 66.7% file descriptors).
BTW: As I tested to find out, the max open files limit is around 2549 in a PacketTunnelProvider extension.
@clowwindy Awesome! Any chance of using BSD, MIT, or MPL 2.0 for that so we can ship it with the Tor VPN? ;)
@chrisballinger
Oh, I meant I would build Shadowsocks directly on lwip. It will be an adapter from a tun device to a Shadowsocks protocol client, without implementing Socks5 protocol.
It turns out that lwip's API is very easy to use. tun2socks.c has demonstrated how to create a tcp listener on a virtual interface and how to operate on connections, etc. Apart from code that reads the command line arguments, it's only a few hundreds lines. We can just fit these code in Shadowsocks or Tor's event loop and bridge them together.
I'll update when I've made any progress.
@clowwindy Ah ok, that's great news. Thank you for all of your amazing work on shadowsocks!
Installed Xcode 7 beta 4 in Yosomite, not getting NetworkExtension template in XCODE. /System/Library/Frameworks/NetworkExtension.framework/Versions/A/Resources/NEProviderTargetTemplates.pkg is not present in my system too!
Two days ago the police came to me and wanted me to stop working on this. Today they asked me to delete all the code from GitHub. I have no choice but to obey.
I hope one day I'll live in a country where I have freedom to write any code I like without fearing.
I believe you guys will make great stuff with Network Extensions.
Cheers!
Thanks so much for providing such great software.
I hope one day I'll live in a country where I have freedom to write any code I like without fearing.
Love you
Thank you
Thank you
Thanks again.
Thank you
take care
Thank you
Thank you and take care.
What do you mean you have no choice? Which law forbid you from writing software?
Thank you.
Thank You.
