add markdown preprocessor to highlight code-blocks

[nastasi-oq]

This PR add the possiblity to define code-blocks inside class docstrings to document API and
the possibility to highlight them using pygments package.
This is not final version, it is just the first iteration to be sure to avoid waste of time.
What is currently missing are:

• optionality of markdown packages
• test integration

Closes #5374.

[carltongibson]

@nastasi-oq There's two things going on here.

First: the strip() call comes after formatting.dedent() and so:

1. Is unnecessary
2. Is breaking markdown formatting.

Second: Adding highlighting etc.

Can we address those separately? (A test and fix for 1, followed by a proposal for 2.) Thanks!

[nastasi-oq]

First: #5378
For the test I need some coordinate, where I can copy from something similar ?

[carltongibson]

@nastasi-oq: I think this'll be a nice addition.

A basic test case will be needed.

Look in compat:

django-rest-framework/rest_framework/compat.py

Lines 221 to 277 in d54df8c

	# Markdown is optional
	try:
	import markdown
	if markdown.version <= '2.2':
	HEADERID_EXT_PATH = 'headerid'
	LEVEL_PARAM = 'level'
	elif markdown.version < '2.6':
	HEADERID_EXT_PATH = 'markdown.extensions.headerid'
	LEVEL_PARAM = 'level'
	else:
	HEADERID_EXT_PATH = 'markdown.extensions.toc'
	LEVEL_PARAM = 'baselevel'
	def apply_markdown(text):
	"""
	Simple wrapper around :func:`markdown.markdown` to set the base level
	of '#' style headers to <h2>.
	"""
	extensions = [HEADERID_EXT_PATH]
	extension_configs = {
	HEADERID_EXT_PATH: {
	LEVEL_PARAM: '2'
	}
	}
	md = markdown.Markdown(
	extensions=extensions, extension_configs=extension_configs
	)
	return md.convert(text)
	except ImportError:
	apply_markdown = None
	markdown = None
	try:
	import pygments
	from pygments.lexers import get_lexer_by_name
	from pygments.formatters import HtmlFormatter
	def pygments_highlight(text, lang, style):
	lexer = get_lexer_by_name(lang, stripall=False)
	formatter = HtmlFormatter(nowrap=True, style=style)
	return pygments.highlight(text, lexer, formatter)
	def pygments_css(style):
	formatter = HtmlFormatter(style=style)
	return formatter.get_style_defs('.highlight')
	except ImportError:
	pygments = None
	def pygments_highlight(text, lang, style):
	return text
	def pygments_css(style):
	return None

Here you'll need to adjust the ordering slightly: the markdown block will need to check for pygments etc. Create a function here to be called in render_markdown:

django-rest-framework/rest_framework/templatetags/rest_framework.py

Lines 69 to 73 in d54df8c

	@register.simple_tag
	def render_markdown(markdown_text):
	if not markdown:
	return markdown_text
	return mark_safe(markdown.markdown(markdown_text))

Let me know if you need input!

[carltongibson]

@nastasi-oq I'm going to close this as-is. I am very happy to take a PR adding this, in line with the notes above. I don't though want to track it as an open issue. Thanks for the input: Hope to see more. 🙂