This is for the more open Internet.
It’s…
@geekbb’s introduction to Warp (https://twitter.com/geekbb/status/1663443373544583168). Although Warp has been out for a long time, it doesn’t have iClaud Private Relay to protect IP privacy, and I don’t need magic to go online. But I suddenly realized that I still had to hide the IP demand.
In the years in which RSSHUb was developed, I found that there were very few public API sites, and many of them would also use strict anti-crawl controls to limit access to their platform content. Some sites block too many requests from the same IP, while others fully block IP addresses of common cloud servers. It has therefore become very difficult to obtain only the latest updates.
[lord-of-the-rings-my-precious] (./attachments/bafybeiaqqbchfi5zs4kfxjnuhbktmyx3v3jz5g6uodgeaql2d5kf54e4tu.gif)
This situation requires the use of agents, but specialized reptile agents are usually expensive and have very low value for money, and it would be great if the unlimited flow of Cloudflare WARP and the rich IP resources could be used by RSSHUb. RSSHUb already supports a generic proxy agreement, provided that the WARP is packaged as a generic proxy.
[image] (./attachments/bafybeidnorcybhjb7lapu5y6zjtnzzffsyrdlfm3fhzcnhtr37u4e677he.png)
Although official clients cannot be easily used directly in the command line environment, such an easy-to-be idea must have been fulfilled. I found an sealed Docker on GitHub.
https://github.com/Mon-ius/Docker-Warp-Socks
Then just add this service to the RSShub <<PROTECTED_BLONK_2> to enable the proxy service.
♪ Protected ♪
Finally add a PROXY_URI environment variable to RSSHUb to use proxy
♪ Protected ♪
I have selected a hootkdeals route that I often use for testing. The site will shield all Digital Ocean IPs and thus remain in 403 status.
[image] (./attachments/bafkreihjpcghhfk4nghyczjb2z56zfzf6ompyq3msr7tsdqjx4tlipsxri.png)
Plus the WARP, we’ll get to it.
[image] (./attachments/bafybeiazr6b66xmkpkw5w7ws57nvnmpj2tjx2genfyqq6ibkejafsrpsz4.png)
In addition, I found that every time I restarted the WRP, I would export a new IP. Although I don’t have time to verify, I feel that the IP should be changed on a regular basis, which is good news for solving the problem of back-crawling.
[image] (./attachments/bafkreihsp2oel6zxmvlnkqvezvlwjltnj65iymnodxkqhrvhw6bsftb4pq.png)
The configuration of WireGuard can also be further defined, including the use of a fee-based version of WARP+ and a custom endpoint to obtain a possible better outcome.
Generate WireGuard profiles available
WARP+ flow and filter endpoint available
https://github.com/fscarmen/warp
It has been argued that there is no significant difference in the velocity of WARP+ ([WARP, WARP+ velocity comparison, and WARP velocity ceilings] (https://www.nodeseek.com/post-9449-1)), but further verification of the impact on the anti-crawling effect is needed.
If everything goes well, many of the strict anti-crawling routes in the official RSSHub case should be reusable. I will do this in a few days and update it here.
[image] (./attachments/bafybeibkqr2di6wp7u5tzllu5cd2zwrnirwxhetyeq63s2xl6fruuail5a.png)