C# Webclient Returning Error 404

I'm using below script to retrieve HTML from an URL. string webURL = @'https://nl.wiktionary.org/wiki/' + word.ToLower(); using (WebClient client = new WebClient()

Solution 1:

try this. You can catch the 404 error content in a try catch block.

var word = Console.ReadLine();
        string webURL = @"https://nl.wiktionary.org/wiki/" + word.ToLower();
        using (WebClient client = new WebClient() {  })
        {
            try
            {

                string htmlCode = client.DownloadString(webURL);

            }
            catch (WebException exception)
            {
                string responseText=string.Empty;

                var responseStream = exception.Response?.GetResponseStream();

                if (responseStream != null)
                {
                    using (var reader = new StreamReader(responseStream))
                    {
                        responseText = reader.ReadToEnd();
                    }
                }

                Console.WriteLine(responseText);
            }
        }

        Console.ReadLine();

Solution 2:

Since this WIKI-server use case-sensitive url mapping, just don't modify case of URL to harvest (remove ".ToLower()" from you code).

Ex.: Lower case:https://nl.wiktionary.org/wiki/categorie:onderwerpen_in_het_nynorsk Result: HTTP 404(Not Found)

Normal (unmodified) case:https://nl.wiktionary.org/wiki/Categorie:Onderwerpen_in_het_Nynorsk Result: HTTP 200(OK) Also, keep in mind what most (if not all) WiKi servers (including this one) generates custom 404 pages, so in browser they looks like "normal" pages, but, despite this, they are serving with 404 http code.