Partager via

Common Q&A about TMG URL Filtering database

  • Article

URL filtering is one of Forefront TMG’s most popular features. The feature makes use of a cloud service, also known as Microsoft Reputation Services (MRS) for URL categorization purposes. In this post we’d like to address some of the more frequent questions we’ve received regarding the URL filtering database and the cloud service.

What is the URL categorization cloud service?

MRS can be thought of as a web service, providing secure access to a huge, cloud-based dynamic repository of URLs and their respective categories. The database features over 70 categories ranging from security-oriented selections like Malicious sites, through productivity-oriented categories such as Online Communities, and ending with liability-oriented categories like Pornography. The database spans tens of millions of unique URLs and their respective categories. Whenever a user behind TMG tries to access a URL, TMG can look up its category by issuing an online query to the cloud service. If you’d like to learn more about URL filtering and the cloud service, check out this post.

What are the sources of the URL filtering database?

The database merges data from several providers. The data providers include internal Microsoft sources as well as 3rd party sources. It is important to note different providers employ different URL categorization techniques. Some employ manual classification while others rely heavily on web crawlers performing automated classification. The highly advanced automated classification techniques ensure TMG URL filtering is as competitive as it gets when it comes to the coverage of the web.

How frequently is the URL filtering database updated?

The database is constantly updated in order to cope with the dynamic nature of the web. New URLs are added, obsolete ones are removed and categories can change based on the page’s content. The update frequency varies per data provider and per URL category.

How good is the coverage for URLs outside the US?

The URL filtering database ensures a global coverage, and is meant to serve TMG customers all over the world. The coverage is regularly monitored against indicators such as Alexa's top Million URL list. In addition, telemetry data gathered from TMG deployments around the world is used to refine the database and improve the coverage for URLs that are popular with our customers.

Is URL categorization performed based on the top level domain only or based on the full URL?

The categorization is based on the full path. This means http://www.contoso.com and http://www.contoso.com/sports could be assigned with different categories in the database.

Can a URL have more than a single category?

Yes. You can learn more about TMG’s support for multiple URL categories in this post.

Can TMG block adult text, images and videos from search results by web search engines?

Yes. You can learn more about TMG’s safe search enforcement support in this post.

 

 

Author:

Dotan Elharrar, Senior Program Manager, Forefront TMG

Reviewers:

David Strausberg, Technical Writer, Forefront TMG

Zakie Mashiah, Group Manager, Forefront TMG

Gabriel Koren, Forefrong TMG test team

Comments

  • Anonymous
    January 01, 2003
    the URL categorization service is not opened currently for products outside Forefront. [Dotan Elharrar, Senior Program Manager, Forefront TMG]

  • Anonymous
    November 16, 2010
    is it possible to access the web service interface of the URL categorization service to utilize it in products other than Forefront?

  • Anonymous
    January 08, 2011
    Does this new TMG web filtering feature allow for quota assignment to users? Will be be able to offer 100% of the current fucntionalities of Websense?

  • Anonymous
    February 10, 2014
    The comment has been removed

  • Anonymous
    May 28, 2014
    Open TMG console then go into Web Access Policy, select the Tasks Tab, then click on Configure URL Category Overrides.
    Then click on Add enter the URL then select the category, click apply, then apply your new config and then you should now get access to that site.
    you can test if its working on the Category Query, in the description it should have "User-defined override of default URL Category"

  • Anonymous
    September 04, 2014
    I have a web site its category is unknown in TMG and I want to open the sites related to the subject of that web site(open the category) how do I do that?