io.github.huoshuiai42/huoshui-fetch

Fetches content from a URL. Args: url: The URL to fetch timeout: Request timeout in seconds (default: 30) follow_redirects: Whether to follow redirects (default: True) user_agent: Custom user agent string (optional) Returns: Dictionary containing status_code, content, headers, encoding, and error (if any)

Fetches a URL with custom headers. Args: url: The URL to fetch headers: Dictionary of custom headers timeout: Request timeout in seconds (default: 30) Returns: Dictionary containing status_code, content, headers, encoding, and error (if any)

Converts HTML content to Markdown format. Args: html: HTML content to convert strip_tags: List of tags to strip (default: ['script', 'style', 'meta', 'link']) heading_style: Style for headings - "ATX" or "SETEXT" (default: "ATX") Returns: Dictionary with converted markdown content

Extracts plain text from HTML content. Args: html: HTML content to convert preserve_links: Whether to preserve link URLs in output (default: False) Returns: Dictionary with extracted text

Cleans HTML by removing scripts, styles, and optionally limiting to allowed tags. Args: html: HTML content to clean allowed_tags: List of allowed tags (None = all tags except script/style) Returns: Dictionary with cleaned HTML

Extracts the main article content from a web page. Args: html: HTML content url: Optional URL for resolving relative links Returns: Dictionary with title, content, text_content, excerpt, and short_title

Extracts all links from HTML content. Args: html: HTML content base_url: Base URL for resolving relative links internal_only: Only return links from the same domain (default: False) Returns: Dictionary with list of links containing url, text, title, and rel attributes

Extracts metadata from HTML including title, description, and Open Graph tags. Args: html: HTML content Returns: Dictionary containing title, description, keywords, author, open_graph, twitter_card, etc.

Extracts all images from HTML content. Args: html: HTML content base_url: Base URL for resolving relative image URLs min_width: Minimum image width filter min_height: Minimum image height filter Returns: Dictionary with list of images containing src, alt, title, dimensions, etc.

Extracts structured data (JSON-LD, microdata) from HTML. Args: html: HTML content Returns: Dictionary containing json_ld and microdata if found

Converts JSON data to readable Markdown format. Args: json_data: JSON string to convert Returns: Dictionary with markdown representation of the JSON