例如:在百度搜索中它会根据你的搜索栏中的查询参数查找相应的文章下面模仿百度的请求搜索;
由于百度的搜索时会对url进行编码
例如 https://www.baidu.com/s?wd=‘软件开发’
会转成https://www.baidu.com/s?wd=%E8%BD%AF%E4%BB%B6%E5%BC%80%E5%8F%91
所以在搜索时要对url进行编码
var request = require("request");
request(encodeURI("https://www.baidu.com/s?wd=软件开发"), function (
error,
response,
html
) {
// if (!error && response.statusCode == 200) {
// console.log(sp.GoogleSERP(html));
// }
// console.log(sp.GoogleSERP(html));
console.log(html);
//html:就是爬取的百度在输入查询参数时生成的页面
});
例如2:针对谷歌的搜索时可以使用serp插件
npm install serp
var express = require("express");
var app = express();
const serp = require("serp");
app.get("/", async (req, res, next) => {
try {
var options = {
qs: {
q: "软件开发",
},
num: 10,
delay: 2000, // in ms
};
const links = await serp.search(options);
console.log(links);
} catch (err) {
console.log(err);
}
});
app.listen(5000);
我这里是自已开了一个node服务来发送请求的
links就是我们的搜索到的文章;
links的结构
[ { url:
'https://baike.baidu.com/item/%E8%BD%AF%E4%BB%B6%E5%BC%80%E5%8F%91',
title: '软件开发_百度百科' },
{ url:
'https://baike.baidu.com/item/%E8%BD%AF%E4%BB%B6%E5%BC%80%E5%8F%91%E5%B7%A5%E7%A8%8B%E5%B8%88',
title: '软件开发工程师_百度百科' },
{ url:
'https://zh.wikipedia.org/wiki/%E8%BD%AF%E4%BB%B6%E5%BC%80%E5%8F%91',
title: '软件开发- 维基百科,自由的百科全书' },
{ url:
'https://cjjc.weblio.jp/content/%E8%BD%AF%E4%BB%B6%E5%BC%80%E5%8F%91',
title: '软件开发の意味 - 中国語辞書 - Weblio日中中日辞典' },
{ url: 'https://www.zhihu.com/question/20359791',
title: '学软件开发有哪些基本要求? - 知乎 - 如何回答' },
{ url: 'https://www.infoq.cn/article/uO38fOkym_WuHYmqD7Lu',
title: '一个沉重的问题:软件开发到底还有价值吗?-InfoQ' },
{ url: 'https://juejin.im/post/6844903864219467790',
title: '作为软件开发人员需要的技术技能- 掘金' },
{ url:
'https://www.expeditors.com/zh-hans/technology/software-development',
title: '软件开发 - Expeditors' },
{ url: 'https://www.mscto.com/',
title: '软件开发网: 软件开发技术教程_开发软件培训_软件开发工具_ ...' },
{ url: 'https://halfrost.com/halfrost_2018/',
title: '如何看待软件开发? - 冰霜之地' } ]
I cannot thank you enough for the blog post. Much thanks again. Want more. Gilbertine Tobias Derte
Thankyou for this grand post, I am glad I observed this internet site on yahoo. Saba Brennan Byler
Every weekend i used to visit this web page, because i want enjoyment, since this this web page conations really nice funny material too. Lethia Rowland Steiner
I will recommend your beautiful post site to my friends
It was very useful, I can say it was a useful article for me.
Very informative blog article.Much thanks again. Fantastic.
I randomly stopped by your site but you are really successful
I randomly stopped by your site but you are really successful
Hi to every body, it’s my first visit of this web site; this website carries awesome and genuinely good stuff for readers.
I get very useful information on your page, I feel lucky
That is really attention-grabbing, You are a very skilled blogger.
I’ve joined your rss feed and stay up for in quest of extra of your
great post. Also, I have shared your web site in my social networks
What’s up everyone, it’s my first pay a quick visit at this web page, and post is really fruitful in favor of me, keep up posting these articles or reviews.
A fascinating discussion is definitely worth comment. I think that
you ought to publish more about this issue, it may not be a
taboo subject but typically folks don’t discuss such topics.
To the next! All the best!!
Hello there! I could have sworn I’ve been to this blog before but after browsing through some of the post I realized it’s new to me.
Nonetheless, I’m definitely glad I found it and I’ll be bookmarking and checking back
frequently!
It is in reality a nice and useful piece of information. I am satisfied that you shared this
helpful information with us. Please keep us informed like this.
Thanks for sharing.
Thanks for sharing your thoughts about 都江堰网站网页UI广告平面设计制作开发建设外包.
Regards
I randomly stopped by your site but you are really successful
Superb blog! Do you have any tips for aspiring
writers? I’m hoping to start my own blog soon but I’m a little lost on everything.
Would you propose starting with a free platform like WordPress or go for
a paid option? There are so many options out there that I’m completely confused
.. Any recommendations? Thanks a lot!
I feel very lucky to follow your page