{"id":67,"date":"2010-05-19T21:16:00","date_gmt":"2010-05-19T19:16:00","guid":{"rendered":"http:\/\/dev.bratched.fr\/en\/?p=67"},"modified":"2010-05-19T21:16:00","modified_gmt":"2010-05-19T19:16:00","slug":"linq-to-xml-and-reading-large-xml-files","status":"publish","type":"post","link":"https:\/\/bratched.com\/en\/2010\/05\/19\/linq-to-xml-and-reading-large-xml-files\/","title":{"rendered":"LINQ to XML and reading large XML files"},"content":{"rendered":"

LINQ to XML makes it relatively easy to read and query XML files. For example consider the following XML file:<\/p>\n

\n
<xml version=\"1.0\" encoding=\"utf-8\" ?>\n    <users>\n        <user name=\"User1\" groupid=\"4\" \/>\n        <user name=\"User2\" groupid=\"1\" \/>\n        <user name=\"User3\" groupid=\"3\" \/>\n        <user name=\"User4\" groupid=\"1\" \/>\n        <user name=\"User5\" groupid=\"1\" \/>\n        <user name=\"User6\" groupid=\"2\" \/>\n        <user name=\"User7\" groupid=\"1\" \/>\n    <\/users><\/pre>\n<\/div>\n
Suppose you would like to find all records with groupid > 2. <\/em>You could be tempted to issue the following query:<\/p>\n
\n
XElement doc = XElement.Load(\"users.xml\");\n    var users = from u in doc.Elements(\"user\")\n                where u.Attribute(\"groupid\") != null &&\n                int.Parse(u.Attribute(\"groupid\").Value) > 2\n                select u;\n    Console.WriteLine(\"{0} users match query\", users.Count());<\/pre>\n<\/div>\n
There\u2019s a flaw in this method. XElement.Load<\/a> method will load the whole XML file in memory and if this file is quite big, not only the query might take long time to execute but it might fail running out of memory. If we had some really large XML files we need to buffer through it instead of reading the whole contents into memory. XmlReader<\/a> is a nice alternative to allowing us to have only the current record into memory which could hugely improve performance. We start by defining a User<\/em> class which will be used to represent a single record:<\/p>\n
\n
public class User \n    {\n        public string Name { get; set; }\n        public int GroupId { get; set; }\n    }<\/pre>\n<\/div>\n
Next we extend the XmlReader<\/a> class with the User method:<\/p>\n
\n
public static IEnumerable<User> Users(this XmlReader source)\n    {\n        while (source.Read())\n        {\n            if (source.NodeType == XmlNodeType.Element && \n                source.Name == \"user\")\n            {\n                int groupId;\n                int.TryParse(source.GetAttribute(\"groupid\"), out groupId);\n                yield return new User\n                {\n                    GroupId = groupId,\n                    Name = source.GetAttribute(\"name\")\n                };\n            }\n        }\n    }<\/pre>\n<\/div>\n
And finally we can execute the query:<\/p>\n
\n
using (XmlReader reader = XmlReader.Create(\"users.xml\"))\n    {\n        var users = from u in reader.Users()\n                    where u.GroupId > 2\n                    select u;\n        Console.WriteLine(\"{0} users match query\", users.Count());\n    }<\/pre>\n<\/div>\n
Conclusion: the second approach runs faster and uses much less memory than the first. The difference is noticeable on large XML files. So if you have to deal with large XML files be cautious when using LINQ to XML.<\/p>\n","protected":false},"excerpt":{"rendered":"
LINQ to XML makes it relatively easy to read and query XML files. For example consider the following XML file: <xml version=”1.0″ encoding=”utf-8″ ?> <users> <user name=”User1″ groupid=”4″ \/> <user name=”User2″ groupid=”1″ \/> <user name=”User3″ groupid=”3″ \/> <user name=”User4″ groupid=”1″ \/> <user name=”User5″ groupid=”1″ \/> <user name=”User6″ groupid=”2″ \/> <user name=”User7″ groupid=”1″ \/> <\/users> Suppose […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[263,266],"tags":[284,328],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t