nginx безопасная ссылка иногда возвращает ошибку 403 - PullRequest
Новая
       3

nginx безопасная ссылка иногда возвращает ошибку 403

0 голосов
/ 21 января 2020

В журналах моего сервера я часто нахожу ошибку 403, когда пользователь обращается к файлам mp4, ссылки на которые скрыты с помощью безопасной ссылки в nginx. Большинство пользователей не сталкиваются с такими трудностями. Но только несколько ...

nginx conf: 

    server {
if ( $scheme = "http" ) {
rewrite ^/(.*)$ https://$host/$1 permanent;
}
        listen 80;
        server_name something.su www.something.su;
        return 301 https://$host$request_uri;

    }

server{

server_name_in_redirect off;
        if ($http_user_agent ~ Opera/9.12|facebook|80legs|360Spider|Aboundex|Abonti|Acunetix|^AIBOT|^Alexibot|Alligator|AllSubmitter|Apexoo|^asterias|^attach|^BackDoorBot|^BackStreet|^BackWeb|Badass|Bandit|Baid|Baiduspider|^BatchFTP|^Bigfoot|^Black.Hole|^BlackWidow|BlackWidow|^BlowFish|Blow|^BotALot|Buddy|^BuiltBotTough|^Bullseye|^BunnySlippers|BBBike|^Cegbfeieh|^CheeseBot|^CherryPicker|^ChinaClaw|^Cogentbot|CPython|Collector|cognitiveseo|Copier|^CopyRightCheck|^cosmos|^Crescent|CSHttp|^Custo|^Demon|^Devil|^DISCo|^DIIbot|discobot|^DittoSpyder|Download.Demon|Download.Devil|Download.Wonder|^dragonfly|^Drip|^eCatch|^EasyDL|^ebingbong|^EirGrabber|^EmailCollector|^EmailSiphon|^EmailWolf|^EroCrawler|^Exabot|^Express|Extractor|^EyeNetIE|FHscan|^FHscan|^flunky|^Foobot|^FrontPage|GalaxyBot|^gotit|Grabber|^GrabNet|^Grafula|^Harvest|^HEADMasterSEO|^hloader|^HMView|^HTTrack|httrack|HTTrack|htmlparser|^humanlinks|^IlseBot|Image.Stripper|Sucker|imagefetch|^InfoNaviRobot|^InfoTekies|^Intelliseek|^InterGET|^Iria|^Jakarta|^JennyBot|^JetCar|JikeSpider|^JOC|^JustView|^Jyxobot|^Kenjin.Spider|^Keyword.Density|libwww|^larbin|LeechFTP|LeechGet|^LexiBot|^lftp|^libWeb|^likse|^LinkextractorPro|^LinkScan|^LNSpiderguy|^LinkWalker|msnbot|MSIECrawler|MJ12bot|MegaIndex|^Magnet|^Mag-Net|^MarkWatch|Mass.Downloader|masscan|^Mata.Hari|^Memo|^MIIxpc|^NAMEPROTECT|^Navroad|^NearSite|^NetAnts|^Netcraft|^NetMechanic|^NetSpider|^NetZIP|^NextGenSearchBot|^NICErsPRO|^niki-bot|^NimbleCrawler|^Ninja|^Nmap|nmap|^NPbot|Offline.Explorer|Offline.Navigator|OpenLinkProfiler|^Octopus|^Openfind|^OutfoxBot|Pixray|probethenet|proximic|^PageGrabber|^pavuk|^pcBrowser|^Pockey|^ProPowerBot|^ProWebWalker|^psbot|^Pump|python-requests|^QueryN.Metasearch|^RealDownload|Reaper|^Reaper|^Ripper|Ripper|Recorder|^ReGet|^RepoMonkey|^RMA|scanbot|seoscanners|^Stripper|^Sucker|Siphon|Siteimprove|^SiteSnagger|SiteSucker|^SlySearch|^SmartDownload|^Snake|^Snapbot|^Snoopy|Sosospider|^sogou|spbot|^SpaceBison|^spanner|^SpankBot|Spinn3r|^Sqworm|Sqworm|Stripper|Sucker|^SuperBot|SuperHTTP|^SuperHTTP|^Surfbot|^suzuran|^Szukacz|^tAkeOut|^Teleport|^Telesoft|^TurnitinBot|^The.Intraformant|^TheNomad|^TightTwatBot|^Titan|^True_Robot|^turingos|^URLy.Warning|^Vacuum|^VCI|VidibleScraper|^VoidEYE|^WebAuto|^WebBandit|^WebCopier|^WebEnhancer|^WebFetch|^Web.Image.Collector|^WebLeacher|^WebmasterWorldForumBot|WebPix|^WebReaper|^WebSauger|Website.eXtractor|^Webster|WebShag|^WebStripper|WebSucker|^WebWhacker|^WebZIP|Whack|Whacker|^Widow|Widow|WinHTTrack|^WISENutbot|WWWOFFLE|^WWWOFFLE|^WWW-Collector-E|^Xaldon|^Xenu|^Zade|^Zeus|ZmEu|^Zyborg|SemrushBot|^WebFuck|^MJ12bot|^majestic12|^WallpapersHD|SputnikBot|Crowsnest|PaperLiBot|peerindex|ia_archiver|Slurp|Aport|NING|JS-Kit|rogerbot|BLEXBot|Twiceler|Java|CommentReader|Yeti|BTWebClient|Tagoobot|Ezooms|igdeSpyder|AhrefsBot|Offline|DISCo|netvampire|^Copier|omgili|socialmediascanner|Jooblebot|SeznamBot|Scrapy|CCBot|linkfluence|veoozbot|Leikibot|Seopult|Faraday|hybrid|Go-http-client|SMUrlExpander|SNAPSHOT|getintent|ltx71|Nuzzel|SMTBot|Laserlikebot|facebookexternalhit|mfibot|OptimizationCrawler|crazy|Dispatch|ubermetrics|^HTMLParser|musobot|filterdb|InfoSeek|omgilibot|DomainSigma|SafeSearch|meanpathbot|statdom|spredbot|StatOnlineRuBot|openstat|DeuSu|semantic|postano|Embedly|NewShareCounts|linkdexbot|GrapeshotCrawler|Digincore|NetSeer|help.jp|getprismatic|Ahrefs|ApacheBench|Applebot|archive|BaiduBot|Birubot|bsalsa|Butterfly|Buzzbot|BuzzSumo|CamontSpider|curl|dataminr|DomainTools|DotBot|FairShare|FeedFetcher|FlaxCrawler|FlightDeckReportsBot|FlipboardProxy|FyberSpider|Gigabot|gold\ crawler|InternetSeer|Jakarta|km.ru|kmSearchBot|Kraken|larbin|Lightspeedsystems|Linguee|LinkBot) {
            return 423;
        }


        if ($http_user_agent ~ LinkExchanger|bingbot|msnbot|LinkpadBot|LivelapBot|LoadImpactPageAnalyzer|majestic|Mediatoolkitbot|MetaURI|MLBot|NerdByNature|NjuiceBot|Nutch|OpenHoseBot|Panopta|pflab|PHP/|pirst|PostRank|ptd-crawler|Purebot|PycURL|Python|QuerySeekerSpider|Ruby|SearchBot|SISTRIX|SiteBot|Sogou|solomono|Soup|suggybot|Superfeedr|SurveyBot|SWeb|trendictionbot|TSearcher|ttCrawler|TurnitinBot|TweetmemeBot|UnwindFetchor|urllib|uTorrent|Voyager|WBSearchBot|Wget|WordPress|woriobot|YottosBot|Zeus|zitebot|bingot|mail.ru|tut.by|Br.by|Zubr.com|All.by|Tit.by|21.by|Rambler|Lycos|nigma.ru|Yahoo|alexa.com|archiver|LiveInternet|BegunAdvertising|vkShare|WebArtexBot|Web-Monitoring|Runet-Research-Crawler|SputnikFaviconBot|CNCat|Virusdie|YoudaoBot|WorldSearch|Wotbox|securepoint|Facebot|AltaVista|Custo|Demon|eCatch|WebWhacker|Express|WebPictures|ExtractorPro|FlashGet|GetRight|GetWeb!|Go!Zilla|Go-Ahead-Got-It|rafula|Stripper|Indy|Spider|Vampire|Foto|WebSpider|WebGo|Quester|Twengabot|perl|scan|email|Pyth|PyQ|WebCollector|WebCopy|webcraw|AcoonBot|adbeat_bot|AddThis.com|adidxbot|ADmantX|ExpertSearch|ExpertSearchSpider|extract|F2S|FastSeek|feedfinder|FeedlyBot|finbot|Flamingo_SearchEngine|FlappyBot|flicky|Flipboard|g00g1e|genieo|Genieo|GigablastOpenSource|GozaikBot|grab|GT::WWW|GTB5|Guzzle|harvest|heritrix|HomePageBot|HTTP::Lite|HubSpot|icarus6|id-search|IDBot|IlseBot|Indigonet|integromedb|IRLbot|ISC\ Systems\ iRc\ Search\ 2.1|JobdiggerSpider|JOC\ Web\ Spider|Jorgee|kanagawa|KINGSpider|kmccrew|mailto:craftbot@yahoo.com|AngloINFO|Antelope|BeetleBot|billigerbot|binlar|bitlybot|BLP_bbot|BoardReader|Bolt\ 0|BOT\ for\ JCE|Bot\ casper|CazoodleBot|checkprivacy|chromeframe|Clerkbot|Cliqzbot|clshttp|DTS.Agent|EasouSpider|ecxi|Elmer|ExaleadCloudView|CommonCrawler|comodo|crawler4j|Crawlera|CRAZYWEBCRAWLER|Curious|CWS_proxy|Default\ Browser\ 0|diavol|DigExt|DoCoMo|DotBot|Baidu) {
            return 423;
        }

#set $no_cache 1;


location ~* ^/(composer.phar) {
    deny all;
}

location = /status-inverser {
    stub_status;
}

location ~* ^/(vendor)($|\/) {deny all;}
expires $expires;
### include google-analytics.conf;
#       location / {
#           post_action @GAlog;
#       }
#location = /404.html {
#   post_action @GAlog404;
#}


if ($request_method !~ ^(GET|HEAD|POST)$ ){return 444;}
        server_tokens off;
        #listen  80;
        listen 443 ssl http2 default_server;
        listen [::]:443 ssl http2 ipv6only=on;

        ssl_certificate    /etc/crt/certificate.crt;
        ssl_certificate_key    /etc/crt/private.key;
        server_name something.su www.something.su;

        ssl_session_cache shared:SSL:10m;
        ssl_session_timeout 1h;
        add_header Strict-Transport-Security "max-age=15768000; includeSubDomains" always;
        http2_push_preload on;
        ssl_dhparam /etc/crt/dhparam.pem;
        ssl_prefer_server_ciphers on;
        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK;
        #add_header Strict-Transport-Security max-age=15768000;
        ssl_stapling on;
        ssl_stapling_verify on;

root /var/www/html/something.su;
charset utf8;

location ~ \.(eot|ttf|otf|woff|woff2)$ {
    add_header Access-Control-Allow-Origin *;
    access_log off;
    expires 365d;
}

location ~* ^.+\.(jpg|jpeg|gif|png|ico|css|pdf|ppt|txt|docx|doc|rtf|js|ttf|otf|woff2|eot|svg|woff)$ {
access_log off;
expires 365d;
}
location ~* ^.+\.(jpg|txt|docx|doc|js)$ {
valid_referers none blocked
               something.su www.something.su;

if ($invalid_referer) {
   rewrite   ^/   https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
}
}


location ~ \.mp4$ 
{

aio threads;

valid_referers none blocked
               something.su www.something.su;

    if ($invalid_referer) {
       rewrite   ^/   https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
    }
      mp4;
secure_link $arg_md5,$arg_expires;
secure_link_md5 "$secure_link_expires$uri$remote_addr secure_word";
if ($secure_link = "") { return 403; }
if ($secure_link = "0") { return 410; }

      if ($request_filename ~* ^.*?/([^/]*?)$)
    {
    set $filename $1;
    add_header Content-Disposition "attachment; filename=$filename";
    }
}

location ~ \.flv$ 
{valid_referers none blocked
               something.su www.something.su;

if ($invalid_referer) {
   rewrite   ^/   https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
}
  flv;
}

location ~ \.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/run/php/php7.3-fpm.sock;
    fastcgi_read_timeout 3000;
    index index.php;
}

#rewrite ^/index.php?page=(.*) /$1 permanent;
location ~ /(.*) {

        try_files $uri $uri/ /index.php?page=$1;


   }

location / {
    index index.php index.html index.htm;
    #try_files $uri $uri/ /index.php =404;
    try_files $uri $uri/ /index.php index.php;
}

location @missing {
    rewrite ^ $scheme://$host/index.php permanent;
}


error_page  404  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
error_page  403  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status&ra=$remote_addr;
error_page  405  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
error_page  415  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
error_page  401  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
error_page  402  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;
error_page  500  https://something.su/errorvps?rq=$request&rf=$http_referer&st=$status;






}
##SERVER

PHP код: 

function bsl($path){
    $expires = time() + 36000;//10h
    $userIp = $_SERVER['HTTP_X_FORWARDED_FOR'];

    $p_ = stripos($userIp, ',');
    if ($p_ !== false) {
        $userIp=explode(',',$userIp);
        $userIp=trim($userIp[0]);
    }
    $secret = 'secure_word';
    $path = str_replace('https://something.su', '', $path);
    $md5 = md5("$expires$path$userIp $secret", true);
    $md5 = base64_encode($md5);
    $md5 = strtr($md5, '+/', '-_');
    $md5 = str_replace('=', '', $md5);
    return  "https://something.su{$path}?md5={$md5}&expires={$expires}";
}

Журнал ошибок и использованные агенты: 

2020-01-21 10:28:16 | IP: 109.110.90.50 | Br: Mozilla/5.0 (Linux; U; Android 9; ru-ru; Redmi 7A Build/PKQ1.190319.001) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/71.0.3578.141 Mobile Safari/537.36 XiaoMi/MiuiBrowser/11.3.4-g
403 |REF: https://something.su/download-446-f-6 | RQF: GET /file.mp4?md5=hQXVx4a2wbIBOtIMAGhWYA

2020-01-21 10:44:06 | IP: 62.182.94.xx | Br: Mozilla/5.0 (Linux; U; Android 8.1.0; ru-ru; Redmi Note 5 Build/OPM1.171019.011) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/71.0.3578.141 Mobile Safari/537.36 XiaoMi/MiuiBrowser/11.3.4-g
403 |REF: https://something.su/download-35-f-1 | RQF: GET /file.mp4 HTTP/1.1

2020-01-21 10:45:41 | IP: 84.54.114.xx, 185.26.180.xx | Br: Opera/9.80 (Android; Opera Mini/18.0.2254/163.76; U; ru) Presto/2.12.423 Version/12.16
403 |REF: https://something.su/download-435-f-1 | RQF: GET /file.mp4?md5=djyUqY5N3btlfQYVs9NKwA

2020-01-21 10:44:03 | IP: 213.87.120.xx | Br: Mozilla/5.0 (Linux; arm; Android 6.0.1; SM-G532F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 YaBrowser/19.9.6.88.00 Mobile Safari/537.36
403 |REF: https://something.su/download-iNwdq oFH3bp62pRkgR52xLaqbTjz JElOn2tX7JXPDEvoV6Xa0Ti3I08PonyGLD7M3gJvriVj5TF2e3l5BzUw== | RQF: GET /file.mp4 HTTP/2.0

& expires = 1235478 .. работает по полной ссылке, но отсутствует в файле журнала ...

Для безопасной ссылки я установил время на 10 часов жизнь. Существует вероятность, что в старых браузерах страница просто не обновляет sh, а загружается как есть, если, например, пользователь получает к ней доступ после 10 часов бездействия. Но что-то говорит мне, что есть подвох, и что-то нужно исправить ...

1 Ответ

0 голосов
/ 22 января 2020

в функции bsl Я не передал IP правильно IP ... 

$ip='';    
if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
    $ip = $_SERVER['HTTP_CLIENT_IP'];
} elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
    $ip = $_SERVER['HTTP_X_FORWARDED_FOR'];
} else {
    $ip = $_SERVER['REMOTE_ADDR'];
}
...