Using Jquery 12.1 and MVC 3 what's the best way to handle large datasets(100k+)?

We're using Linq to Entity framework and I'm selecting everything into a viewmodel 

IQueryable<ViewModel> vm = from products in context.vw_Products  select products;

and passing it into the view from our Index Action.

I have remote paging enabled so it doesn't load the entire dataset all at once and the pagingGetData action which returns the same dataset as before.

But this seems to be significantly slower than all the other grids, taking anywhere from 5 to 12 seconds just to finish rendering the grid.

Is there a way to set the grid model's total count and set how many pages it will be so I can utilize the int page and int pagesize I pass into the pagingGetData action? Or what is the best way to do this?